[Live-devel] Frames from the future - or maybe not

[Ross Finlayson]

> *** It is not about the frames arriving in weird order *** , in fact, they arrive in monotonic presentation time order (I just use baseline and I and P frames), like this (number being the presentation timestamp):
> 
> 13  14  15  16  17  18
> 
> All good.  But then I check at what time they arrive to the client (by looking at the client's system clock):
> 
> 13 arrives to client at client time 13.5  .. i.e. it arrives after the presentation time, as one would expect (13<13.5)
> 14 arrives to client at client time 14.5
> 
> .. but
> 
> 15 arrives to client at client time 14.9   .. so it arrives earlier than the presentation time! (15>14.9)
> 16 arrives to client at client time 15.5
> 
> and again, as one would expect
> 
> 17 arrives to client at time 17.1
> etc.
> 
> Trying to synchronize such presentation times with respect to system time (when visualizing the stream) is impossible.
> But maybe I am not supposed to look at the timestamps, just decode and present the frames on the screen as they arrive..

No, you’re supposed to buffer incoming frames for a short period of time (typically a few 100 ms), and then play them back at the proper rate (based on their presentation times).  I.e., you implement a ‘jitter buffer’
	https://en.wikipedia.org/wiki/Jitter#Jitter_buffers

This is the way all network media players work (at least, all network media players that are any good).

Things get a little more complicated if (as sometimes happens, and which I suspect may be happening in your case) the server’s clock runs at a slightly different rate than the client’s clock.  (Remember that the “presentation time”s are generated by the server, using the server’s clock.)  In that case you may find that, over time, your jitter buffer runs dry (if the server’s clock is slower than the client’s), or fills up (if the server’s clock is faster than the client’s).  There are various techniques (of varying sophistication) for dealing with this.  You might want to look at how VLC implements its jitter buffer.


Ross Finlayson
Live Networks, Inc.
http://www.live555.com/