[Live-devel] RTSP audio & video synchronization issue

Mon Sep 15 07:37:29 PDT 2014

Hi all,

I have some doubts on liveMedia regarding RTSP audio & video
synchronization.

We have developed a streaming application using live555 library to stream
via RTSP. It works like charm when streaming only one stream per session
(audio or video). However, problems appear when I try to stream both audio
and video using the same RTSP session.

I tried playing it using VLC and it only plays the video. In debug mode, it
shows this message continuously:

 core audio output warning: buffer too late (-541608 us): dropped

In order to understand this behaviour I used the testRTSPClient as RTSP
client, modified to print also the Normal Play Time. What I could observe
is that Presentation Times of both audio and video streams are respectively
coherent between them. However, after some seconds running, there is an
abrupt change in Presentation Time of one of the streams (usually audio),
which I suppose that corresponds to RTCP synchronization. After that, there
is a big gap between video and audio NPT which remains there during all the
transmission, taking sometimes negative values.

This is the testRTSPClient output (the timestamp gap can be observed):

Stream "rtsp://192.168.10.77:8554/S22J/"; audio/PCMU: Received 1436
bytes. Presentation
time: 1410784970.033291 NPT: 2.911316
Stream "rtsp://192.168.10.77:8554/S22J/"; audio/PCMU: Received 484
bytes. Presentation
time: 1410784970.033291 NPT: 2.911316
Stream "rtsp://192.168.10.77:8554/S22J/"; audio/PCMU: Received 1436
bytes. Presentation
time: 1410784970.053291 NPT: 2.931316
Stream "rtsp://192.168.10.77:8554/S22J/"; audio/PCMU: Received 484
bytes. Presentation
time: 1410784970.053291 NPT: 2.931316
Stream "rtsp://192.168.10.77:8554/S22J/"; audio/PCMU: Received 1436
bytes. Presentation
time: 1410784970.073291 NPT: 2.951316
Stream "rtsp://192.168.10.77:8554/S22J/"; audio/PCMU: Received 484
bytes. Presentation
time: 1410784970.073291 NPT: 2.951316
Stream "rtsp://192.168.10.77:8554/S22J/"; video/H264: Received 1780
bytes. Presentation
time: 1410784970.163530 NPT: 3.041563
Stream "rtsp://192.168.10.77:8554/S22J/"; video/H264: Received 3093
bytes. Presentation
time: 1410784970.163530 NPT: 3.041563
Stream "rtsp://192.168.10.77:8554/S22J/"; video/H264: Received 9393
bytes. Presentation
time: 1410784970.163530 NPT: 3.041563
Stream "rtsp://192.168.10.77:8554/S22J/"; video/H264: Received 5410
bytes. Presentation
time: 1410784970.163530 NPT: 3.041563
Stream "rtsp://192.168.10.77:8554/S22J/"; audio/PCMU: Received 1436
bytes. Presentation
time: 1410784970.093291 NPT: 2.971316
Stream "rtsp://192.168.10.77:8554/S22J/"; audio/PCMU: Received 484
bytes. Presentation
time: 1410784970.093291 NPT: 2.971316
Stream "rtsp://192.168.10.77:8554/S22J/"; audio/PCMU: Received 1436
bytes. Presentation
time: 1410784970.113291 NPT: 2.991316
Stream "rtsp://192.168.10.77:8554/S22J/"; audio/PCMU: Received 484
bytes. Presentation
time: 1410784970.113291 NPT: 2.991316
Stream "rtsp://192.168.10.77:8554/S22J/"; video/H264: Received 1731
bytes. Presentation
time: 1410784977.533251 NPT: 10.411284
Stream "rtsp://192.168.10.77:8554/S22J/"; video/H264: Received 3474
bytes. Presentation
time: 1410784977.533251 NPT: 10.411284
Stream "rtsp://192.168.10.77:8554/S22J/"; video/H264: Received 11707
bytes. Presentation
time: 1410784977.533251 NPT: 10.411284
Stream "rtsp://192.168.10.77:8554/S22J/"; video/H264: Received 8298
bytes. Presentation
time: 1410784977.533251 NPT: 10.411284
Stream "rtsp://192.168.10.77:8554/S22J/"; audio/PCMU: Received 1436
bytes. Presentation
time: 1410784970.133291 NPT: 3.011316
Stream "rtsp://192.168.10.77:8554/S22J/"; audio/PCMU: Received 484
bytes. Presentation
time: 1410784970.133291 NPT: 3.011316
Stream "rtsp://192.168.10.77:8554/S22J/"; audio/PCMU: Received 1436
bytes. Presentation
time: 1410784970.153291 NPT: 3.031316
Stream "rtsp://192.168.10.77:8554/S22J/"; audio/PCMU: Received 484
bytes. Presentation
time: 1410784970.153291 NPT: 3.031316
Stream "rtsp://192.168.10.77:8554/S22J/"; video/H264: Received 2468
bytes. Presentation
time: 1410784977.574917 NPT: 10.452950
Stream "rtsp://192.168.10.77:8554/S22J/"; video/H264: Received 4043
bytes. Presentation
time: 1410784977.574917 NPT: 10.452950
Stream "rtsp://192.168.10.77:8554/S22J/"; video/H264: Received 12999
bytes. Presentation
time: 1410784977.574917 NPT: 10.452950
Stream "rtsp://192.168.10.77:8554/S22J/"; video/H264: Received 9043
bytes. Presentation
time: 1410784977.574917 NPT: 10.452950
Stream "rtsp://192.168.10.77:8554/S22J/"; audio/PCMU: Received 1436
bytes. Presentation
time: 1410784970.173291 NPT: 3.051316
Stream "rtsp://192.168.10.77:8554/S22J/"; audio/PCMU: Received 484
bytes. Presentation
time: 1410784970.173291 NPT: 3.051316
Stream "rtsp://192.168.10.77:8554/S22J/"; audio/PCMU: Received 1436
bytes. Presentation
time: 1410784970.193291 NPT: 3.071316
Stream "rtsp://192.168.10.77:8554/S22J/"; audio/PCMU: Received 484
bytes. Presentation
time: 1410784970.193291 NPT: 3.071316
Stream "rtsp://192.168.10.77:8554/S22J/"; video/H264: Received 3153
bytes. Presentation
time: 1410784977.616583 NPT: 10.494616
Stream "rtsp://192.168.10.77:8554/S22J/"; video/H264: Received 3766
bytes. Presentation
time: 1410784977.616583 NPT: 10.494616
Stream "rtsp://192.168.10.77:8554/S22J/"; video/H264: Received 6609
bytes. Presentation
time: 1410784977.616583 NPT: 10.494616
Stream "rtsp://192.168.10.77:8554/S22J/"; video/H264: Received 4695
bytes. Presentation
time: 1410784977.616583 NPT: 10.494616

I checked the RTP timestamps (in transmission) and everything seems OK (no
gaps and coherent between them).

This behaviour is not the one I would expect and I think it is exactly
what's happening when I try to play the session using VLC. So my question
is what do you think is happening here and how do you think I can solve
this issue? I have the feeling that I do something wrong in transmission
(maybe defining the Presentation Time) but I can't see what.

Thanks in advance for any hint,

Kind regards,
--------------------------------------------------------

 Gerard Castillo Lasheras
  Enginyer de Projectes
  Fundació i2CAT - Unitat Audiovisual
  SkypeID: gerardcl85
  Telf.: +34.93.553.25.48
--------------------------------------------------------
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.live555.com/pipermail/live-devel/attachments/20140915/4506d4e3/attachment.html>