[Live-devel] Layered video with Live555

[Tim Stabrawa]

Ross Finlayson wrote:
> Yes.  Unfortunately, the current stream parsing code discards 
> already-read data once it encounters the end of the input file. 
> There are probably ways to work around this (e.g., pass some function 
> other than "FramedSource::handleClosure" to the parent constructor 
> when constructing "MPEGVideoStreamParser"), but it would be messy. 
> Instead, I suggest just ignoring the issue if you can.  (Is the very 
> last NAL unit in each file really important to get?)  If you can't 
> ignore it, then you could also work around the problem by appending a 
> 'start code' to the end of each file, before you start reading it.
>   
Of the two sample files I'm working with right now, one ends with a 
suffix NAL unit, and the other actual slice data.  So, the effect of 
dropping that would either be the preceeding frame gets sent on the base 
layer accidentally or the last chunk of slice data is never received.  I 
suppose I could deal with either if they happened, but if it can be 
fixed, why not?  I tried just putting a fake start code at the end of 
the file and that seems to do the trick (the received file is the same 
as the sent one, sans fake start code).  That's good enough for now, I'd 
say.  (It's for a demonstration, after all.)
> By the way, the *real* difficulty that you're going to face in 
> writing your "H264VideoStreamFramer" subclass is computing proper 
> presentation times and durations (the "fPresentationTime" and 
> "fDurationInMicroseconds" member variables) for each NAL unit.  (This 
> has tripped up other people who have tried streaming H.264 video from 
> a file.
For my purposes, I plan on just computing these based on the average 
bitrate of the file.  Not ideal, but should get the job done.

That gets me to thinking though ... When it comes time to re-combine the 
streams on the client side, I assume the best (only?) way is to use the 
presentation times to put things back together correctly.  
Unfortunately, I don't see any "Mux" classes to base this off of.  In 
fact, the only reordering code I see is the ReorderingPacketBuffer 
class, but this looks like it keys off of RTP sequence numbers, not 
presentation times.  Have you ever heard of someone needing to do this?  
What would you suggest?

- Tim