[Live-devel] Multiple NALUs concatenated together in one afterGettingFrame

[Deanna Earley]

Hello Ross.

I now have a packet capture of the second case I mentioned (H.264 I frame being dropped) which I think is related.
https://dl.dropboxusercontent.com/u/2931731/Bugs/liveMedia/dropped%20I%20frame/videostream.pcapng
https://dl.dropboxusercontent.com/u/2931731/Bugs/liveMedia/dropped%20I%20frame/frames2.LOG
If you look at the frame2.log, you will see a “frame num gap” being reported by FFmpeg on line 421.
The 2 lines immediately before it are P frames with a frame number of 14, then 1, but the intermediary I frame was never passed from liveMedia to our file sink.

https://dl.dropboxusercontent.com/u/2931731/Bugs/liveMedia/dropped%20I%20frame/rawframe/425%201430469392.350150.dat
https://dl.dropboxusercontent.com/u/2931731/Bugs/liveMedia/dropped%20I%20frame/rawframe/426%201430469392.425538.dat
https://dl.dropboxusercontent.com/u/2931731/Bugs/liveMedia/dropped%20I%20frame/rawframe/427%201430469392.425538.dat
https://dl.dropboxusercontent.com/u/2931731/Bugs/liveMedia/dropped%20I%20frame/rawframe/428%201430469392.507293.dat
These 4 files represent each frame passed to our sink, P frame 14, an SPS, a PPS, and a P frame (with no intermediary I frame).
Comparing these to the packet capture, the SPS and PPS (file 426 and 427) are in packet 19288, with the I frame at packet numbers 19289 to 19463 where the marker bit is set.
The P frame in file 428 can be seen in the packet capture starting at packet 19511.

For reference, here is the top of our afterGettingFrame method in our file sink:
void H264VideoMemorySink::afterGettingFrame(unsigned frameSize, unsigned int numTruncatedBytes, struct timeval presentationTime) {

      char str[512];
      sprintf_s(str, 512, "c:\\debug\\rawframe\\%d %u.%06u.dat", ++framex, presentationTime);
      FILE* file = fopen(str, "wb");
      fwrite(fBuffer, 1, frameSize, file);
      fclose(file);

As far as I can see, the data in the network stream is correct and framed correctly.
(A test tool isn’t flagging any errors about oversized SPS/PPS frames, or unexpected ordering of H.264 frames.)

Do you have any further thoughts on what could be going wrong?
I've spent a while looking at the code and can’t see anything obvious in liveMedia or our code.

Note that our original problem was liveMedia joining several frames together before sending them through to our sink, but that only occurs very infrequently.
The dropped I frames occurs every few minutes and seems to be in the same area of the code (parsing H.264 frames).

I’m still investigating here and will post if I find anything else out.

Many thanks

--
Deanna Earley | Lead developer | icatchercctv

w: www.icode.co.uk/icatcher<http://www.icode.co.uk/icatcher> | t: 01329 835335 | f: 01329 835338
Registered Office : 71 The Hundred, Romsey, SO51 8BZ. Company Number : 03428325

From: live-devel [mailto:live-devel-bounces at ns.live555.com] On Behalf Of Ross Finlayson
Sent: 05 May 2015 09:44
To: LIVE555 Streaming Media - development & use
Subject: Re: [Live-devel] Multiple NALUs concatenated together in one afterGettingFrame

Thanks, I was trying to get something concrete by analysing the network stream.
This camera is unusual (compared to others we have) in that it sends an SPS and a PPS before every I frame.
The SPS/PPS share an RTP frame in a STAP with no marker bit, while the I/P frames are in FU-A packets (with the marker set on the last RTP frame).

That looks correct.  As far as I can tell, our code (“H264VideoRTPSource.cpp”) should correctly be delivering each NAL unit - one at a time - from this input stream.

Ross Finlayson
Live Networks, Inc.
http://www.live555.com/

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.live555.com/pipermail/live-devel/attachments/20150506/af9033ad/attachment.html>