[Live-devel] MP4 video trak samples: were do 'non-frame' NAL units belong?

[Muhammad Hassan]

U can write your own source filter using live555 client side code and
connect it to any muxer filter like matroska, mp4 or avi and then save it.
Feel free to contact me at hassan.nust at gmail.com
On Mar 19, 2013 10:48 PM, "Joel Keller" <jkeller at miovision.com> wrote:

> Greetings experts,
>
> Live555 is playing a key role in our industrial machine vision application
> - thank you for the library!  I am experiencing a problem saving MP4 format
> video files using openRTSP.  The RTSP/RTP stream is coming from a camera,
> and I can successfully save the 'raw/elementary' H264 NAL stream with
> openRTSP.  However, when I save the video as MP4, with something like this:
>
> openRTSP -d 30 -b 1000000 -B 1000000 -4 -y -w 1536 -h 1536 -f 15 rtsp://
> 10.0.3.11/live.sdp > myvideo.mp4
>
>
> I find that the generated MP4 file contains NAL units in the 'mdat' box
> which do not correspond to video frames, specifically NAL units of type
> SPS, PPS, and SEI.  These three NAL units are periodically interspersed
> with the frame data.  This interspersion itself is of course not a problem
> as the media data area can contain non-frame data, and this meta data is of
> course needed to interpret the frame.  However these NAL units are treated
> as 'MP4 samples' in the MP4 'Sample Table Boxes'.  Specifically, in the
> stts (Decoding time to sample box), these samples have a "delta" of zero,
> implying that their presentation time is the same as the samples
> which precede them (this makes sense, because they are not frames of the
> video, but it violates ISO spec, which states that "The sample entries are
> ordered by decoding time stamps; therefore the deltas are all
> non-negative.").   I find that this causes some (most?) video players to
> "stutter" when playing back the MP4 file (avidemux2 for example),  because
> they determine the video's frame rate and then go sample-by-sample through
> the video.  When the player gets to these non-frame samples, it appears to
> delay for the frame-duration, but it cannot decode the frame (because the
> sample is not a frame).  This leads to a video with ~15 smooth frames, then
> 3 pause frames, then ~15 smooth frames.
>
> My questions are:
>
> 1)  What is the correct thing to do with these NAL units in the MP4 file?
>
> 2)  How can I make openRTSP/live555 create an MP4 file without this
> stuttering?
>
>
>
> Possible answers I have to #1 are:
>
> a.  "non-frame" NAL units are appropriate samples in MP4 video traks.  But
> then what to do with the stts deltas?
>
> b.  "non-frame" NAL units should not show up as samples (see avcC box),
> and should be 'skipped over' based on MP4 'chunks'.  But how do I get
> live555 to do this?
>
> c.  "non-frame' NAL units should be "combined" their following "frame" NAL
> unit to be a single 'MP4 sample'.  (ie a sample is not necessarily a single
> NAL unit). Again how would I get live555 to do this?
>
>
>
> Thank you for reading this long post.  I am most grateful for any help.
>
>
> Thanks,
>
> Joel Keller
>
>
> _______________________________________________
> live-devel mailing list
> live-devel at lists.live555.com
> http://lists.live555.com/mailman/listinfo/live-devel
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.live555.com/pipermail/live-devel/attachments/20130319/eb50849f/attachment-0001.html>