[Live-devel] Does live555 exist a way to sync video/audio streams?

[Eric Hsieh]

Hi Ross,

Thanks to provide a lot of important info to me.

One more question on sync topic. Thanks.
If the worst case is
RTSP server receive request from client, the time is 1.576000
Server start finding the latest H264/AAC frames.
H264 frame's presentation time start from 1.000000.
AAC frame's presentation time start from 1.640000.

When client receive the first H264 frame to play, and receive the first AAC frame and play.
You can see video without audio for 640ms.

What do you think how to fix this issue?
If we want to sync streams in same session, how can we do?
Please advise. Thanks.
________________________________
寄件者: live-devel <live-devel-bounces at us.live555.com> 代表 Ross Finlayson <finlayson at live555.com>
寄件日期: 2020年8月17日 上午 11:01:36
收件者: LIVE555 Streaming Media - development & use <live-devel at us.live555.com>
主旨: Re: [Live-devel] Does live555 exist a way to sync video/audio streams?

This message was sent from outside of the company. Please do not click links or open attachments unless you recognize the source of this email and know the content is safe.


> On Aug 17, 2020, at 12:27 PM, Eric Hsieh <Eric.Hsieh at liteon.com> wrote:
>
> And one question for sync video/audio stream.
>
> Background:
> Our system will prepare audio/video frames to memory, always capture frames from driver.
> For video, it is H264, FPS is 30 and driver output one I frame per second.
> For audio, it is AAC, 16KHz.
>
> Behavior:
> When player ask RTSP streaming from our server.
> For video stream, we go to memory and find latest I frame to send.
> For audio stream, we go to memory and find latest frame to send.
> On worst case, first timestamp of video frame <= first timestamp of audio frame + 1(second).
>
> Question:
> We want to sync video/audio streams when player ask a stream.
> Video/audio streams are belonged to same ServerMediaSession.
> Is it possible to pass the first timestamp of video frame to audio stream?
> Is there a way to sync data between FramedSource?

Yes.  As we note here
        https://apc01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flive555.com%2FliveMedia%2Ffaq.html%23separate-rtp-streams&data=02%7C01%7Ceric.hsieh%40liteon.com%7Ce7ca4d78837840c4323a08d8425bbe02%7C5a7a259b6730404bbc255c6c773229ca%7C0%7C0%7C637332309099735559&sdata=sY8he2rxbSe%2FvoRoAPzsBfg0bJGy3ag1lpqRZtekyC0%3D&reserved=0
the LIVE555 code does this automatically, *provided that*:
        1/ Accurate presentation times (i.e., the “fPresentationTime” variable) are set each time your “FramedSource” subclass delivers a frame (i.e., when implementing “doGetNextFrame()”, and
        2/ These presentation times are aligned to ‘wall clock’ time - i.e., the time that you’d get by calling “gettimeofday()”.

Note that you should continue to use the term “presentation time” rather than “timestamp”.  (The term “timestamp” can be confusing, because RTP has ’timestamps’, but they are used only internally by the RTP/RTCP protocol, and are not visible to server (or client) application code.)

Ross Finlayson
Live Networks, Inc.
https://apc01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.live555.com%2F&data=02%7C01%7Ceric.hsieh%40liteon.com%7Ce7ca4d78837840c4323a08d8425bbe02%7C5a7a259b6730404bbc255c6c773229ca%7C0%7C0%7C637332309099735559&sdata=ZhItrgtPDWYrj48hGceALCQKhTf7O3FUmCjFQe%2BoFc0%3D&reserved=0


_______________________________________________
live-devel mailing list
live-devel at lists.live555.com
https://apc01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.live555.com%2Fmailman%2Flistinfo%2Flive-devel&data=02%7C01%7Ceric.hsieh%40liteon.com%7Ce7ca4d78837840c4323a08d8425bbe02%7C5a7a259b6730404bbc255c6c773229ca%7C0%7C0%7C637332309099735559&sdata=oI9KWW2tvuJFpq1WmcX6v4%2B%2FbEx86sJVGH8RmDYVYgE%3D&reserved=0

Confidential Information:This message is sent to the intended recipient and may contain privileged or confidential information. If you received this transmission in error, please notify the sender with a replying e-mail and delete the message and any attachment.Transmission Caveat and Virus Alert: Internet communications cannot be guaranteed to be timely, secure, error or virus-free. The sender does not accept liability for any errors or omissions.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.live555.com/pipermail/live-devel/attachments/20200817/721319b9/attachment-0001.htm>