[Live-devel] H264 RTP stream recording to AVI

[jon.bird at gd-ms.uk]

Hi,

I've been investigating the Live555 library with a view to using it in an embedded platform for (initially) the recording of H264 video received over RTP into AVI files. I don't have a great deal of knowledge about the innards of H264, I'm hoping that in using this library I won't need to... To date we've been using gstreamer pipelines to accomplish this, they work well enough but I think are overkill for what we want to do here and don't quite offer the same level of flexibility that I think this library offers.

My initial approach has been to look at the testMPEG2TransportReceiver sample, I've successfully adapted this to capture one of our streams and dump it to a file however this isn't playable or recognizable by tools like vlc or ffmpeg. Changing the output sink from FileSink to H264VideoFileSink seems to improve things slightly but still doesn't yield any playable video.

So rather than using our existing video sources, I've been looking at using the OpenRTSP client to record streams from testOnDemandRTSPServer. I have a test H264 file in a Matroska container and can (sort of) successfully record a "raw" H264 file, playable by VLC using the following:

openRTSP -b 400000 -d 25 rtsp://192.168.204.3:8854/matroskaFileTest

I say "sort of" because I only get between 12-15s worth of video (when I specified 25s) and for the first half of the stream, the colours appear very much washed out, which may be related to I-Frames (or the lack of them possibly). 

Now I try capturing to AVI:

openRTSP -b 400000 -d 25 -i -w 1080 -h 720 -f 50 rtsp://192.168.204.3:8854/matroskaFileTest > test.avi

This creates a file however it doesn't contain a valid index & even after having VLC rebuild the index or play "as is", just results in a spew of error messages on the console.

The MP4 version:

openRTSP -b 400000 -d 25 -4 -w 1080 -h 720 -f 50 rtsp://192.168.204.3:8854/matroskaFileTest > test.mp4

gives results similar to the "raw" H264 variant.

It looks to me then as if the AVI multiplexor isn't working properly, at least for H264 anyway. For us, this isn't a major problem because we already have code that can create AVI files, I just need to get the H264 frames into the correct format for storing into the file. 

Looking at the code for openRTSP, a key difference between that and what I've done in the testMPEG2TransportReceiver is the passing of sPropParameterSetsStr:

          fileSink = H264VideoFileSink::createNew(*env, outFileName,
                                                  subsession->fmtp_spropparametersets(),
                                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

                                                  fileSinkBufferSize, oneFilePerFrame);

which injects (additional NALs?) into the start/end of the frames. This comes from the session data and by defaulting this to NULL in the openRTSP code gives me similar results to my initial runs with testMPEG2TransportReceiver ie. an unplayable raw file. So this is clearly key (and may well be I guess what is missing in part with the AVI creation).

So my guess is that (as a minimum) I need to feed in the appropriate NALs into H264VideoFileSink::createNew (based on the incoming data) in order to get something suitable for storing into media files. However given the other quirks I'm seeing here suggests to me that there may be a number of other things I need to do/take into account which aren't clear from the examples and documentation that is online.

If anyone has any helpful suggestions here, it would be appreciated.

Thanks,


Jon.

--
Jon Bird, CEng MBCS
Software Engineer
Electronic Systems
General Dynamics United Kingdom Ltd.
Castleham Road, St Leonards on Sea, East Sussex, TN38 9NJ

Telephone: +441424798278
Email: jon.bird at gd-ms.uk
Website: www.generaldynamics.uk.com      


------------------------------------------------------------------------------

This email and any files attached are intended for the addressee and may contain information of a confidential nature. If you are not the intended recipient, be aware that this email was sent to you in error and you should not disclose, distribute, print, copy or make other use of this email or its attachments. Such actions, in fact, may be unlawful. In compliance with the various Regulations and Acts, General Dynamics United Kingdom Limited reserves the right to monitor (and examine for viruses) all emails and email attachments, both inbound and outbound. Email communications and their attachments may not be secure or error- or virus-free and the company does not accept liability or responsibility for such matters or the consequences thereof. General Dynamics United Kingdom Limited, Registered Office: 21 Holborn Viaduct, London EC1A 2DY. Registered in England and Wales No: 1911653.