[Live-devel] Layered video with Live555

[Ross Finlayson]

>For an academic demonstration, I'm planning on extending Live555 to
>support RTP transport of scalable H.264 video and was hoping someone
>with a reasonable amount of experience with Live555 could help steer me
>in the direction of least pain ...
>
>Basically, I'll be using the reference codec for H.264 SVC (currently in
>development) to generate a file containing H.264 NAL units.  The
>important difference between the output of this codec and a standard
>H.264 stream is the addition of two NAL unit types (20 & 21), which
>carry information about which layer of video is described in the
>preceding/current NAL unit.  For now, assume I know how to parse this
>file and determine which NAL units belong to which layers.  My intention
>is to send each layer out either multiplexed in the same RTP stream (the
>easy way) or in separate RTP streams (the hard / interesting way),
>according to this draft RFC:
>http://www.ietf.org/internet-drafts/draft-ietf-avt-rtp-svc-00.txt

This is interesting.  I suggest proceeding in three steps (with each 
step requiring additional work building on the previous steps):
1/ Stream regular (non-SVC) H.264 video from a file.  You will be 
able to test this using VLC.
2/ Add additional SVC layers, multiplexed in the same RTP stream as 
the base layer.
2/ Use separate RTP streams for separate SVC layers.

>So, from everything I've read so far, there doesn't seem to be any Real
>Code I have to write in the H.264 RTP classes - aside from setting up
>the correct SDP parameters, etc.

That's basically true, I think.  However, the implementation of these 
classes may need to be updated to handle interleaved mode (as defined 
in RFC 3984) - something that we don't yet implement, but which (upon 
a cursory look at the SVC Internet-Draft) appears to be necessary for 
carrying SVC.

>  What I'm not sure about though is how
>to hook up a single file parser class (that I'd have to write) to one or
>more RTP Sink classes.

If you're streaming on a single RTP stream (steps 1/ or 2/), then 
it's fairly straightforward: You'll need to write your own subclass 
of "H264VideoStreamFramer"; that subclass will parse the input stream 
(from a "ByteStreamFileSource").  You'll then 'play' this to a 
"H264VideoRTPSink" object.

For step 3/, however:

>   Is it as simple as just instantiating multiple
>instances of the H264VideoRTPSink class and including them as separate
>sub-sessions in my ServerMediaSession?

Yes.

>   How would I indicate to the
>Live555 framework which sub-session a given NAL unit belongs to?

You would need to write a 'demultiplexor' class that splits the 
incoming NAL stream (coming out of your 'Framer') into several 
different sub-streams of NAL units - one for each outgoing 
"H264VideoRTPSink".  (But note that you'll need this only for step 
3/.)

-- 

Ross Finlayson
Live Networks, Inc.
http://www.live555.com/