[Live-devel] H264 video conflict with AAC audio using same ServerMediaSession

[Kevin Chen]

Hi All,

I implemented live H264 Video and aac Audio source from hardware encoder,
and then rewrite "H264framedsource" and "AACFramedsource" class derive from
"FramedSource" based on the "DevSource", both audio and video are using
"singalNewFrameData" as callback function to signal the "deliverFrame"
function as soon as new frame data were ready, audio and video work fine
when they are separated.

Right now, I have problem make them stream out together, it seems audio
will hang the video immediately when connect the RTSP server using VLC, I
can only hear audio without video, for sometimes, just video without audio,
the rtsp server is implement based on "testH264VideoStreamer.cpp", I set
new rtp and rtcp Groupsock and rtcp instance and RTP payload value for aac
audio, but they use the same multicast address, the sink for video is
"H264videoRTPSink" and "MPEG4GenericRTPSink" for audio,  both are added to
the same sms's subsesssion using "PassiveServerMediaSubsession" subsession.

I doubt the problem that the time stamp for video and audio are not
correct, since during playing using VLC, I notice that two different time
are shown alternately, for the video, I set the "fPresentationTime" as
below: since the frame rate of my H264 video is 30fps, the time for the
first video frame is based on the "gettimeofday", then increased based on
frame rate

----------

  if( encoder_video_ts.tv_sec == 0 && encoder_video_ts.tv_usec == 0 )

    gettimeofday( &encoder_video_ts, NULL ); //Set the first stream time
stamp from wall clock

  else {

    unsigned uSeconds = encoder_video_ts.tv_usec + 1000000/30; //Framerate
is 30fps

    encoder_video_ts.tv_sec += uSeconds/1000000;

    encoder_video_ts.tv_usec = uSeconds%1000000;

  }

 fPresentationTime = encoder_video_ts.tv_sec;

----------
And for the AAC audio, for each frame data, I set "fPresetationTime" from
"gettimeofday".

since audio and video are feeding using two separated thread, is this the
correct way to sync the A/V ?

Thank you.

Kevin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.live555.com/pipermail/live-devel/attachments/20150506/6cf93af4/attachment.html>