[Live-devel] RTP timestamp issue after RTCP synchronization

[Bruno Marchand]

Hi!
I am working on a RTSP recorder and proxy module using the live library.  Basically, it is a module which streams from an Axis camera to a matroska file and supply a RTSP server for viewers.  It is very similar to the software "live555ProxyServer" but with a matroska muxer.

The camera generates two RTP streams, one H264 and one AAC.

As far as I know, the live pipeline is the same as the one in "live555ProxyServer" with an added StreamReplicator just after the rtpSources.  For each stream, the replicator streams in the RTSP server and in the matroska muxer.  I had to create a few components for the matroska muxer and to "fit" the stream replicator in the RTSP server.

At first, instead of a RSTP server, I created RTP sinks and connected VLC using a SDP.  It worked well but letting VLC select its listening ports and adding RTCP appeared to me to be a good idea.  I replaced RTP sinks by a RTSP server which worked well until I added the audio stream.  In fact, the server properly stream data to VLC and VLC plays fine until the RTCP synchronisation occurs. It then spits these messages to the console:

[00000000026b4bf8] main video output warning: picture is too late to be displayed (missing 131234696 ms)
[00000000026b4bf8] main video output warning: picture is too late to be displayed (missing 131234664 ms)
[00000000026b4bf8] main video output warning: picture is too late to be displayed (missing 131234632 ms)
[00000000026b4bf8] main video output warning: picture is too late to be displayed (missing 131234600 ms)
[00000000026b4bf8] main video output warning: picture is too late to be displayed (missing 131234568 ms)

As you certainly already know, VLC uses live for rtp/rtsp streaming, so I tried to watch these streams using a program based on "testRTSPclient".  The program displays the packets timestamps and adds an exclamation mark when not synchronized:

Audio 2633490369 !
Video 2633490390 !
Video 2633490423 !
Audio 2633490433 !
Video 2633490457 !
Video 2633490490 !
Audio 2751915938
Video 2633490523 !
Video 2633490556 !
Audio 2751916002
Video 2633490590 !
Video 2633490623 !
Audio 2751916066
Video 2633490656 !
Video 2633490689 !
Audio 2751916130
Video 2633490723 !
Audio 2751916194
Video 2633490756 !
Video 2633490789 !
Audio 2751916258
Video 2633490823 !
Video 2633490856 !
Video 2633490889 !
Audio 2751916322
Audio 2751916386
Video 2633490922 !
Video 2633490956 !
Video 2633490989 !
Audio 2751916450
Video 2633491022 !
Video 2620681379
Audio 2751916514
Video 2620681413
Video 2620681446
Audio 2751916578
Video 2620681479
Video 2620681513
Audio 2751916642
Video 2620681546
Video 2620681579
Audio 2751916706
Video 2620681612
Video 2620681646
Audio 2751916770
Video 2620681679
Audio 2751916834
Video 2620681712
Video 2620681746
Video 2620681779
Audio 2751916890

As you can see, the timestamps (in milli sec) are in the same range before synchronization, but after synchronization, the gap VLC is complaining about appears.
The gap is around 2751916890 - 2620681779 = 131235111.

I'm still searching why RTCP create this gap.  The issue is not the timestamp gap in one stream, but the difference between the synchronized timestamps of the two streams.
The issue can be reproduced with an unmodified "live555ProxyServer".

The SDP, exchanged through RTSP is
v=0
o=- 1347046461812023 1 IN IP4 172.20.22.20
s=Session streamed by "VideoRecorder"
i=av_stream
t=0 0
a=tool:LIVE555 Streaming Media v2012.07.06
a=type:broadcast
a=control:*
a=range:npt=0-
a=x-qt-text-nam:Session streamed by "VideoRecorder"
a=x-qt-text-inf:av_stream
m=video 0 RTP/AVP 96
c=IN IP4 0.0.0.0
b=AS:50000
a=rtpmap:96 H264/90000
a=fmtp:96 packetization-mode=1;profile-level-id=420029;sprop-parameter-sets=Z0IAKeKQGQJvy4C3AQEBpB4kRUA=,aM48gA==
a=control:track1
m=audio 0 RTP/AVP 97
c=IN IP4 0.0.0.0
b=AS:32
a=rtpmap:97 MPEG4-GENERIC/16000
a=fmtp:97 streamtype=5;profile-level-id=15;mode=aac-hbr;sizelength=13;indexlength=3;indexdeltalength=3;config=1408
a=control:track2

Any help will be appreciated.
Thanks,

Bruno Marchand
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.live555.com/pipermail/live-devel/attachments/20120907/18ff036b/attachment-0001.html>