[Live-devel] Zero Latency Live Streaming

[Ross Finlayson]

First, you haven’t said specifically how you are using the “LIVE555 Streaming Media” software.  (Remember that the software includes a RTSP server, a RTSP client, a RTSP proxy server, and many other things.)  But from the context of your message, I assume that you are using it as a RTSP server, to stream (software-encoded) H.264 video over RTSP/RTP to a RTSP/RTP client application (perhaps not using our software) that then decodes and renders the video.

I doubt anyone has time to look through your code, but you haven’t said how much latency you are talking about, and how you know (or think) that it’s occurring in our code.  It’s rather rude and presumptuous to assume that the latency is occurring in our code, without considering and measuring all of the possible places where it might be occurring:
1/ In your H.264 encoder - i.e., between the creation of a raw video frame, and the time that it is encoded into (one or more) H.264 NAL units.
2/ Between the time that a H.264 NAL unit is created, and the time that it is transmitted within a RTP packet.
3/ Between the time that the RTP packet is transmitted (by your server application), and the time that the transmitted packet leaves your server machine.
4/ Between the time that the transmitted RTP packet leaves your server machine, and the time that it arrives at your client machine.
5/ Between the time that the RTP packet arrives at the client machine, and the time that it is received by the client application “omxplayer”.
6/ Between the time that “omxplayer” receives (all of the) H.264 NAL units for a frame, and the time that the frame is rendered on the client machine’s screen.

Only 2/ is the responsibility of our server software - but I can tell you right now that the latency here is insignificant.  As is usually the latency in 4/ (assuming that you’re streaming over a LAN, rather than a WAN).  So your latency is most likely a combination of 1/, 3/, 5/, and 6/ - which only you can figure out and fix.

But note that any latency in 3/ is caused by the server machine’s OS, and any latency in 5/ is caused by the client machine’s OS.  Each of these latencies is usually low, assuming that you’re streaming your RTP packets over UDP.  (I assume that you’re not doing something stupid like streaming over TCP instead.)  But again, this is only something that you can figure out.

My first guess is that most of your latency (whatever it is) is occurring in 6/ - i.e., within your “omxplayer” client decoder/player application.  (These client player apps sometimes have a ‘jitter buffer’ latency parameter that you can tune.)  But again, nothing to do with us.


Ross Finlayson
Live Networks, Inc.
http://www.live555.com/