[Live-devel] track synchronization with live555

[David Betrand]

Hello Ross,

This mail will again talk about synchronization issues, sorry about that ;-)

I know live555 library uses RTCP SR reports to synchronize audio and video streams, but doesn't use the rtp-info header present in the PLAY response.

I will be quite direct : I think you shouldn't, and should follow section 12.33 from RFC2326 and section 14.38 from draft-ietf-mmusic-rfc2326bis-14.txt instead. Those standards clearly state that the only way to guarantee synchronization is to use 'rtptime' timestamps from the rtp-info header. I think that the misunderstanding here is that the library doesn't make the difference between NPT and NTP times. Yes, RTCP SRs can be used to map RTP timestamps into NTP wall clock, but only 'rtptime' can be used to map RTP timestamps into the NPT time.

The main problems with relying on SR is that you need at least one SR from each track before you can do this, and not all implementations send SRs right away (note also that immediate SRs are effectively nothing much
different than RTP-Info values, though the values actually used would
likely be different). The other problem is that data can be delayed in
the network at different rates meaning even if the source isnt doing
this intentionally, it could happen anyway.
Let's take the case for which I'm currently in trouble using live555 library: my application is a RTSP client connecting to a live encoder (Envivio M2 to be accurate) that can be accessed via RTSP. Here is the sequence of packets I receive from the encoder, for the audio track(AMR, 8000 Hz sampling)  :

t0 : reception of rtp-info : RTP timestamp = 11146
t0 + 3.7 sec : reception of first RTP packet : RTP timestamp = 32373
t0 + 3.9 sec : reception of second RTP packet : RTP timestamp = 33970
t0 + 4 sec : reception of first RTCP SR : RTP timestamp = 43150
t0 + 4.1 sec : reception of next RTP packet : RTP timestamp = 35570
 

So,
-if you look at event #1 and #3 in this sequence, this seems fine : 4 seconds elapse and the timestamps increase with (4 * 8000) = 32000
-if you look at events #2 #3 and #5 in this sequence, this seems fine too because the RTP timstamp differences are typical for AMR (1600 timestamps, for 200 ms)

But :
- if you look at event #1 and #2, you see that almost 4 seconds elapse but the RTP timestamps only increase with 21200 while we could expect a value around 30000. 
--> look at the timestamp jump between event#3 and event #4 : very big !
--> look at the timstamp "jump back" between event #4 and event #5 : a negative value representing approximatively 1 second ! 

--> ALL those jumps will be reflected in RTPReceptionStats::fSyncTime and the resultPresentationTime will of course follow those jumps too ...
 
The "uncommon" thing in those timestamps is that the synchronizations points given in the rtp-info' and in the SR are not multiplexed in time sequence with the surrounding RTP packets : it is like the audio packets have to be played in the past ! As unexpected it might be, I don't think it is actually illegal in any way. If you've seen something in the relevant specs that makes you disagree, of course let me know ...

So, my opinion is that only the rtp-info header shouldbe used to map the RTP timestamps to a common point in time (corresponding to NPT in PLAY response). On the other hand, SRs MAY be used to furthermore compute the difference between the receiver wall clock and the sender wal clock in case of long presentations.

I don't know why but I've the feeling that you won't agree easily ... Am I right ? 

Looking forward to sharing this with U

David

-- 
_______________________________________________
Surf the Web in a faster, safer and easier way:
Download Opera 9 at http://www.opera.com

Powered by Outblaze