[Live-devel] H264 video conflict with AAC audio using same ServerMediaSession

[Ross Finlayson]

> Right now, I have problem make them stream out together, it seems audio will hang the video immediately when connect the RTSP server using VLC

I suggest that you first use our “openRTSP” RTSP client application <http://www.live555.com/openRTSP/ <http://www.live555.com/openRTSP/>> to test receiving both audio and video from your server.  Using “openRTSP”, you should see (nonempty) files for both the audio and video stream.  (“openRTSP” is a much simpler client application than VLC, because it doesn’t render the incoming media, and doesn’t care whether their presentation times are correct.)


> I doubt the problem that the time stamp for video and audio are not correct, since during playing using VLC, I notice that two different time are shown alternately, for the video, I set the "fPresentationTime" as below: since the frame rate of my H264 video is 30fps, the time for the first video frame is based on the "gettimeofday", then increased based on frame rate  
> 
> ----------
>   if( encoder_video_ts.tv_sec == 0 && encoder_video_ts.tv_usec == 0 )
> 
Are you certain that this branch is actually getting executed - i.e., that your “encoder_video_ts” variable (both the “tv_sec” and “tv_usec” fields) is actually initialized to 0?

>     gettimeofday( &encoder_video_ts, NULL ); //Set the first stream time stamp from wall clock                                          
> 
>   else {
> 
>     unsigned uSeconds = encoder_video_ts.tv_usec + 1000000/30; //Framerate is 30fps
> 
This is correct *only if* each NAL unit that you deliver is a complete ‘access unit’ - i.e., picture.  If, instead, your stream includes PPS and SPS NAL units, and/or ‘slice’ NAL units, then this calculation won’t be correct.

In any case, I suggest simplifying your code by just setting “fPresentationTime” for *each* of your video NAL units by calling “gettimeofday()” - as you already do for the audio frames.

> since audio and video are feeding using two separated thread, is this the correct way to sync the A/V ?

Yes, if the “fPresentationTime” values are accurate, *and* aligned with ‘wall clock’ time (i.e., times that would be generated by “gettimeofday()”), then audio and video will get properly synchronized by standards-conforming RTP/RTCP clients (like VLC).

Ross Finlayson
Live Networks, Inc.
http://www.live555.com/


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.live555.com/pipermail/live-devel/attachments/20150506/b1b0cc08/attachment.html>