A simple way to get video in and out of GNU Radio

One of the things I want to do with GNU Radio and the USRP is video transmissions over radio. For this purpose I need a way to read video sources – including files, webcams and other video capture devices – and to display or save it on the other end.

I suppose the right way to do this is to create specific signal sources and sinks for GNU Radio. This can be done either by “direct access”, i.e. read the UVC device directly, or by using a higher level library like libvlc or the ffmpeg libraries (libav*). The latter has indeed been used for audio and the code is available from the Comprehensive GNU Radio Archive (CGRAN) under Mediatools.

For this experiment, however, I decided to try something simpler that that I can try and conclude within an evening: Use VLC as capture  and playback applications and connect to GNU Radio using either the TCP or UDP interfaces.

After a minimal amount of reading I concluded that I have to use UDP unicast. Multicast might work too but I don’t really care about that at the moment. According to the VLC streaming matrix, UDP limits the available containers to MPEG-TS and Raw. Having no idea what Raw is (I mean in bits and bytes) I chose TS with H.264 video and AAC audio codecs.

On the GNU Radio side, I created a simple flow graph where an UDP source is connected to an UDP sink. The UDP source will be connected to the VLC server, while the UDP sink to the VLC client. Obviously, this GNU Radio flow graph doesn’t do anything interesting besides routing one network port to another, but if the routing works the video is passed in and out of GNU Radio successfully and would thus also be usable in a more functional flow graph.

Simple GNU Radio Companion flow graph
.
UDP source settings in GRC	UDP sink settings in GRC

Some quirks of this setup:

1. Both the client and the server must be running before the GNU Radio flow graph is started.
2. The GNU Radio flow graph stops working if either the client or the server is stopped

I would think that both of the above problems can be solved if needed. The problem is that a GNU Radio flowgraph needs a continuous stream of data (and a way to get rid of the data) so breaking the connection at either end will naturally cause it to hang unless such cases are handled properly.

There are also immediate advantages of this simple setup:

1. Can read and write and format, codec and device supported by VLC
2. Server, GNU Radio and client can run on separate computers (though this could also be achieved without using VLC/UDP)
3. Really easy to set up

I might also try using the ffmpeg+mplayer combination but I assume it will work the same way. I can say though, that VLC doesn’t seem to be very good to read the UVC video source. Often, it can not open the device and even when it can, I found it difficult to change parameters, e.g. the capture resolution. With mplayer I had no trouble capturing any of the supported sizes from 320×240 to 1600×1200.

Edit 2010.03.29: A very similat experiment using ffmpeg/mplayer has been done in Wireless H.264 Video Testbed by Eric Qin Chen and his students.