[Live-devel] Synchronizing audio and video streams
Severin Schoepke
severin.schoepke at gmail.com
Mon Jun 4 10:32:39 PDT 2007
Hello again!
I investigated a little deeper and here is what I came up with: Just to
recap: I have two threads, one is reading audio and video frames and
stores them in two queues. Another thread reads these queues, encodes
the frames to MPEG4 and MP3, and streams them out via live555. The
live555 part is organized as follows: I have two FramedSource subclasses
that provide encoded audio and video frames respectively. These are
connected to an MPEG4VideoFramer and an MPEG1or2AudioFramer
respectively, which are connected to an MPEG4ESVideoRTPSink and an
MPEG1or2AudioRTPSink. To provide the stream, I use a DarwinInjector (I
tested your RTSPServer and it yielded in the same results).
My problem is, that the video source's doGetNextFrame() is called much
less than the audio source's. As a result, the video source's queue gets
filled with video frames, that are not consumed immediately. Therefore,
the video stream begins to lag behind the audio stream.
Here is some debug output (sorry, it has to be that detailed and long
for it to make sense):
Beginning streaming...
Beginning to read from video input...
2007-06-04 15:36:58.101 Mischpult[12008] 1180964218:69929 - buffering
audio frame (now 1 audio frames in queue)
2007-06-04 15:36:58.107 Mischpult[12008] 1180964218:102202 - buffering
audio frame (now 2 audio frames in queue)
2007-06-04 15:36:58.133 Mischpult[12008] 1180964218:118806 - buffering
video frame (now 1 video frames in queue)
2007-06-04 15:36:58.138 Mischpult[12008] 1180964218:135415 - buffering
audio frame (now 3 audio frames in queue)
2007-06-04 15:36:58.152 Mischpult[12008] 1180964218:151894 -
doGetNextFrame(): provided Video frame: fFrameSize = 16505 bytes with
fPresentationTime = 1180964218:118806 (now 0 Video frames in queue)
2007-06-04 15:36:58.178 Mischpult[12008] 1180964218:168648 - buffering
video frame (now 1 video frames in queue)
2007-06-04 15:36:58.179 Mischpult[12008] 1180964218:168648 - buffering
audio frame (now 4 audio frames in queue)
2007-06-04 15:36:58.206 Mischpult[12008] 1180964218:201928 - buffering
audio frame (now 5 audio frames in queue)
2007-06-04 15:36:58.207 Mischpult[12008] 1180964218:206957 -
doGetNextFrame(): provided Video frame: fFrameSize = 10751 bytes with
fPresentationTime = 1180964218:168648 (now 0 Video frames in queue)
Beginning to read from audio input...
2007-06-04 15:36:58.209 Mischpult[12008] 1180964218:208258 -
doGetNextFrame(): provided Audio frame: fFrameSize = 384 bytes with
fPresentationTime = 1180964218:69929 (now 4 Audio frames in queue)
2007-06-04 15:36:58.209 Mischpult[12008] 1180964218:208463 -
doGetNextFrame(): provided Audio frame: fFrameSize = 384 bytes with
fPresentationTime = 1180964218:102202 (now 3 Audio frames in queue)
2007-06-04 15:36:58.209 Mischpult[12008] 1180964218:208654 -
doGetNextFrame(): provided Audio frame: fFrameSize = 384 bytes with
fPresentationTime = 1180964218:135415 (now 2 Audio frames in queue)
Play this stream (from the Darwin Streaming Server) using the URL:
2007-06-04 15:36:58.226 Mischpult[12008] 1180964218:218542 - buffering
video frame (now 1 video frames in queue)
rtsp://127.0.0.1/test.sdp
2007-06-04 15:36:58.238 Mischpult[12008] 1180964218:235192 - buffering
audio frame (now 3 audio frames in queue)
2007-06-04 15:36:58.239 Mischpult[12008] 1180964218:238262 -
doGetNextFrame(): provided Audio frame: fFrameSize = 384 bytes with
fPresentationTime = 1180964218:168648 (now 2 Audio frames in queue)
2007-06-04 15:36:58.239 Mischpult[12008] 1180964218:238564 -
doGetNextFrame(): provided Audio frame: fFrameSize = 384 bytes with
fPresentationTime = 1180964218:201928 (now 1 Audio frames in queue)
2007-06-04 15:36:58.239 Mischpult[12008] 1180964218:238838 -
doGetNextFrame(): provided Audio frame: fFrameSize = 384 bytes with
fPresentationTime = 1180964218:235192 (now 0 Audio frames in queue)
2007-06-04 15:36:58.275 Mischpult[12008] 1180964218:268453 - buffering
video frame (now 2 video frames in queue)
2007-06-04 15:36:58.276 Mischpult[12008] 1180964218:268453 - buffering
audio frame (now 1 audio frames in queue)
2007-06-04 15:36:58.291 Mischpult[12008] 1180964218:290250 -
doGetNextFrame(): provided Video frame: fFrameSize = 33815 bytes with
fPresentationTime = 1180964218:218542 (now 1 Video frames in queue)
2007-06-04 15:36:58.304 Mischpult[12008] 1180964218:301744 - buffering
audio frame (now 2 audio frames in queue)
2007-06-04 15:36:58.324 Mischpult[12008] 1180964218:323193 -
doGetNextFrame(): provided Video frame: fFrameSize = 18718 bytes with
fPresentationTime = 1180964218:268453 (now 0 Video frames in queue)
2007-06-04 15:36:58.327 Mischpult[12008] 1180964218:326644 -
doGetNextFrame(): provided Audio frame: fFrameSize = 384 bytes with
fPresentationTime = 1180964218:268453 (now 1 Audio frames in queue)
2007-06-04 15:36:58.327 Mischpult[12008] 1180964218:326893 -
doGetNextFrame(): provided Audio frame: fFrameSize = 384 bytes with
fPresentationTime = 1180964218:301744 (now 0 Audio frames in queue)
2007-06-04 15:36:58.328 Mischpult[12008] 1180964218:318377 - buffering
video frame (now 2 video frames in queue)
2007-06-04 15:36:58.338 Mischpult[12008] 1180964218:334998 - buffering
audio frame (now 1 audio frames in queue)
2007-06-04 15:36:58.338 Mischpult[12008] 1180964218:337320 -
doGetNextFrame(): provided Audio frame: fFrameSize = 384 bytes with
fPresentationTime = 1180964218:334998 (now 0 Audio frames in queue)
2007-06-04 15:36:58.366 Mischpult[12008] 1180964218:365940 -
doGetNextFrame(): provided Video frame: fFrameSize = 15993 bytes with
fPresentationTime = 1180964218:318377 (now 0 Video frames in queue)
2007-06-04 15:36:58.376 Mischpult[12008] 1180964218:368262 - buffering
video frame (now 1 video frames in queue)
2007-06-04 15:36:58.376 Mischpult[12008] 1180964218:368262 - buffering
audio frame (now 1 audio frames in queue)
2007-06-04 15:36:58.405 Mischpult[12008] 1180964218:401508 - buffering
audio frame (now 2 audio frames in queue)
2007-06-04 15:36:58.406 Mischpult[12008] 1180964218:405368 -
doGetNextFrame(): provided Video frame: fFrameSize = 11362 bytes with
fPresentationTime = 1180964218:368262 (now 0 Video frames in queue)
2007-06-04 15:36:58.406 Mischpult[12008] 1180964218:405875 -
doGetNextFrame(): provided Audio frame: fFrameSize = 384 bytes with
fPresentationTime = 1180964218:368262 (now 1 Audio frames in queue)
2007-06-04 15:36:58.407 Mischpult[12008] 1180964218:406090 -
doGetNextFrame(): provided Audio frame: fFrameSize = 384 bytes with
fPresentationTime = 1180964218:401508 (now 0 Audio frames in queue)
2007-06-04 15:36:58.425 Mischpult[12008] 1180964218:418148 - buffering
video frame (now 1 video frames in queue)
2007-06-04 15:36:58.438 Mischpult[12008] 1180964218:434783 - buffering
audio frame (now 1 audio frames in queue)
2007-06-04 15:36:58.438 Mischpult[12008] 1180964218:437275 -
doGetNextFrame(): provided Audio frame: fFrameSize = 384 bytes with
fPresentationTime = 1180964218:434783 (now 0 Audio frames in queue)
2007-06-04 15:36:58.463 Mischpult[12008] 1180964218:462300 -
doGetNextFrame(): provided Video frame: fFrameSize = 6914 bytes with
fPresentationTime = 1180964218:418148 (now 0 Video frames in queue)
2007-06-04 15:36:58.475 Mischpult[12008] 1180964218:468062 - buffering
video frame (now 1 video frames in queue)
2007-06-04 15:36:58.475 Mischpult[12008] 1180964218:468062 - buffering
audio frame (now 1 audio frames in queue)
2007-06-04 15:36:58.480 Mischpult[12008] 1180964218:479619 -
doGetNextFrame(): provided Audio frame: fFrameSize = 384 bytes with
fPresentationTime = 1180964218:468062 (now 0 Audio frames in queue)
2007-06-04 15:36:58.503 Mischpult[12008] 1180964218:501314 - buffering
audio frame (now 1 audio frames in queue)
2007-06-04 15:36:58.504 Mischpult[12008] 1180964218:503167 -
doGetNextFrame(): provided Audio frame: fFrameSize = 384 bytes with
fPresentationTime = 1180964218:501314 (now 0 Audio frames in queue)
2007-06-04 15:36:58.526 Mischpult[12008] 1180964218:517955 - buffering
video frame (now 2 video frames in queue)
2007-06-04 15:36:58.537 Mischpult[12008] 1180964218:534568 - buffering
audio frame (now 1 audio frames in queue)
2007-06-04 15:36:58.537 Mischpult[12008] 1180964218:536853 -
doGetNextFrame(): provided Audio frame: fFrameSize = 384 bytes with
fPresentationTime = 1180964218:534568 (now 0 Audio frames in queue)
2007-06-04 15:36:58.576 Mischpult[12008] 1180964218:567849 - buffering
video frame (now 3 video frames in queue)
2007-06-04 15:36:58.576 Mischpult[12008] 1180964218:567849 - buffering
audio frame (now 1 audio frames in queue)
2007-06-04 15:36:58.576 Mischpult[12008] 1180964218:575914 -
doGetNextFrame(): provided Audio frame: fFrameSize = 384 bytes with
fPresentationTime = 1180964218:567849 (now 0 Audio frames in queue)
2007-06-04 15:36:58.604 Mischpult[12008] 1180964218:601117 - buffering
audio frame (now 1 audio frames in queue)
2007-06-04 15:36:58.604 Mischpult[12008] 1180964218:603868 -
doGetNextFrame(): provided Audio frame: fFrameSize = 384 bytes with
fPresentationTime = 1180964218:601117 (now 0 Audio frames in queue)
2007-06-04 15:36:58.625 Mischpult[12008] 1180964218:617749 - buffering
video frame (now 4 video frames in queue)
2007-06-04 15:36:58.637 Mischpult[12008] 1180964218:634375 - buffering
audio frame (now 1 audio frames in queue)
2007-06-04 15:36:58.637 Mischpult[12008] 1180964218:636793 -
doGetNextFrame(): provided Audio frame: fFrameSize = 384 bytes with
fPresentationTime = 1180964218:634375 (now 0 Audio frames in queue)
2007-06-04 15:36:58.674 Mischpult[12008] 1180964218:667639 - buffering
video frame (now 5 video frames in queue)
2007-06-04 15:36:58.675 Mischpult[12008] 1180964218:667639 - buffering
audio frame (now 1 audio frames in queue)
2007-06-04 15:36:58.677 Mischpult[12008] 1180964218:676360 -
doGetNextFrame(): provided Audio frame: fFrameSize = 384 bytes with
fPresentationTime = 1180964218:667639 (now 0 Audio frames in queue)
2007-06-04 15:36:58.703 Mischpult[12008] 1180964218:700917 - buffering
audio frame (now 1 audio frames in queue)
2007-06-04 15:36:58.703 Mischpult[12008] 1180964218:702700 -
doGetNextFrame(): provided Audio frame: fFrameSize = 384 bytes with
fPresentationTime = 1180964218:700917 (now 0 Audio frames in queue)
2007-06-04 15:36:58.725 Mischpult[12008] 1180964218:717556 - buffering
video frame (now 6 video frames in queue)
2007-06-04 15:36:58.737 Mischpult[12008] 1180964218:734194 - buffering
audio frame (now 1 audio frames in queue)
2007-06-04 15:36:58.737 Mischpult[12008] 1180964218:736303 -
doGetNextFrame(): provided Audio frame: fFrameSize = 384 bytes with
fPresentationTime = 1180964218:734194 (now 0 Audio frames in queue)
2007-06-04 15:36:58.775 Mischpult[12008] 1180964218:767466 - buffering
video frame (now 7 video frames in queue)
2007-06-04 15:36:58.776 Mischpult[12008] 1180964218:767466 - buffering
audio frame (now 1 audio frames in queue)
2007-06-04 15:36:58.777 Mischpult[12008] 1180964218:776165 -
doGetNextFrame(): provided Audio frame: fFrameSize = 384 bytes with
fPresentationTime = 1180964218:767466 (now 0 Audio frames in queue)
2007-06-04 15:36:58.804 Mischpult[12008] 1180964218:800710 - buffering
audio frame (now 1 audio frames in queue)
2007-06-04 15:36:58.804 Mischpult[12008] 1180964218:803513 -
doGetNextFrame(): provided Audio frame: fFrameSize = 384 bytes with
fPresentationTime = 1180964218:800710 (now 0 Audio frames in queue)
2007-06-04 15:36:58.824 Mischpult[12008] 1180964218:817343 - buffering
video frame (now 8 video frames in queue)
2007-06-04 15:36:58.836 Mischpult[12008] 1180964218:833977 - buffering
audio frame (now 1 audio frames in queue)
2007-06-04 15:36:58.836 Mischpult[12008] 1180964218:835898 -
doGetNextFrame(): provided Audio frame: fFrameSize = 384 bytes with
fPresentationTime = 1180964218:833977 (now 0 Audio frames in queue)
2007-06-04 15:36:58.874 Mischpult[12008] 1180964218:867232 - buffering
video frame (now 9 video frames in queue)
2007-06-04 15:36:58.874 Mischpult[12008] 1180964218:867232 - buffering
audio frame (now 1 audio frames in queue)
2007-06-04 15:36:58.874 Mischpult[12008] 1180964218:873908 -
doGetNextFrame(): provided Audio frame: fFrameSize = 384 bytes with
fPresentationTime = 1180964218:867232 (now 0 Audio frames in queue)
2007-06-04 15:36:58.903 Mischpult[12008] 1180964218:900515 - buffering
audio frame (now 1 audio frames in queue)
2007-06-04 15:36:58.903 Mischpult[12008] 1180964218:902707 -
doGetNextFrame(): provided Audio frame: fFrameSize = 384 bytes with
fPresentationTime = 1180964218:900515 (now 0 Audio frames in queue)
2007-06-04 15:36:58.925 Mischpult[12008] 1180964218:917143 - buffering
video frame (now 10 video frames in queue)
2007-06-04 15:36:58.936 Mischpult[12008] 1180964218:933803 - buffering
audio frame (now 1 audio frames in queue)
2007-06-04 15:36:58.936 Mischpult[12008] 1180964218:935908 -
doGetNextFrame(): provided Audio frame: fFrameSize = 384 bytes with
fPresentationTime = 1180964218:933803 (now 0 Audio frames in queue)
2007-06-04 15:36:58.975 Mischpult[12008] 1180964218:967039 - buffering
video frame (now 11 video frames in queue)
2007-06-04 15:36:58.975 Mischpult[12008] 1180964218:967039 - buffering
audio frame (now 1 audio frames in queue)
2007-06-04 15:36:58.976 Mischpult[12008] 1180964218:975325 -
doGetNextFrame(): provided Audio frame: fFrameSize = 384 bytes with
fPresentationTime = 1180964218:967039 (now 0 Audio frames in queue)
2007-06-04 15:36:59.004 Mischpult[12008] 1180964219:318 - buffering
audio frame (now 1 audio frames in queue)
2007-06-04 15:36:59.004 Mischpult[12008] 1180964219:3354 -
doGetNextFrame(): provided Audio frame: fFrameSize = 384 bytes with
fPresentationTime = 1180964219:318 (now 0 Audio frames in queue)
2007-06-04 15:36:59.024 Mischpult[12008] 1180964219:16939 - buffering
video frame (now 12 video frames in queue)
2007-06-04 15:36:59.036 Mischpult[12008] 1180964219:33566 - buffering
audio frame (now 1 audio frames in queue)
2007-06-04 15:36:59.037 Mischpult[12008] 1180964219:36807 -
doGetNextFrame(): provided Audio frame: fFrameSize = 384 bytes with
fPresentationTime = 1180964219:33566 (now 0 Audio frames in queue)
2007-06-04 15:36:59.073 Mischpult[12008] 1180964219:66841 - buffering
video frame (now 13 video frames in queue)
2007-06-04 15:36:59.074 Mischpult[12008] 1180964219:66841 - buffering
audio frame (now 1 audio frames in queue)
2007-06-04 15:36:59.078 Mischpult[12008] 1180964219:77372 -
doGetNextFrame(): provided Audio frame: fFrameSize = 384 bytes with
fPresentationTime = 1180964219:66841 (now 0 Audio frames in queue)
2007-06-04 15:36:59.102 Mischpult[12008] 1180964219:100110 - buffering
audio frame (now 1 audio frames in queue)
2007-06-04 15:36:59.103 Mischpult[12008] 1180964219:102081 -
doGetNextFrame(): provided Audio frame: fFrameSize = 384 bytes with
fPresentationTime = 1180964219:100110 (now 0 Audio frames in queue)
2007-06-04 15:36:59.124 Mischpult[12008] 1180964219:116753 - buffering
video frame (now 14 video frames in queue)
2007-06-04 15:36:59.136 Mischpult[12008] 1180964219:133379 - buffering
audio frame (now 1 audio frames in queue)
2007-06-04 15:36:59.136 Mischpult[12008] 1180964219:135859 -
doGetNextFrame(): provided Audio frame: fFrameSize = 384 bytes with
fPresentationTime = 1180964219:133379 (now 0 Audio frames in queue)
2007-06-04 15:36:59.174 Mischpult[12008] 1180964219:166671 - buffering
video frame (now 15 video frames in queue)
2007-06-04 15:36:59.175 Mischpult[12008] 1180964219:166671 - buffering
audio frame (now 1 audio frames in queue)
2007-06-04 15:36:59.175 Mischpult[12008] 1180964219:174519 -
doGetNextFrame(): provided Audio frame: fFrameSize = 384 bytes with
fPresentationTime = 1180964219:166671 (now 0 Audio frames in queue)
2007-06-04 15:36:59.203 Mischpult[12008] 1180964219:199908 - buffering
audio frame (now 1 audio frames in queue)
2007-06-04 15:36:59.203 Mischpult[12008] 1180964219:202622 -
doGetNextFrame(): provided Audio frame: fFrameSize = 384 bytes with
fPresentationTime = 1180964219:199908 (now 0 Audio frames in queue)
2007-06-04 15:36:59.224 Mischpult[12008] 1180964219:216545 - buffering
video frame (now 16 video frames in queue)
2007-06-04 15:36:59.235 Mischpult[12008] 1180964219:233167 - buffering
audio frame (now 1 audio frames in queue)
2007-06-04 15:36:59.236 Mischpult[12008] 1180964219:235221 -
doGetNextFrame(): provided Audio frame: fFrameSize = 384 bytes with
fPresentationTime = 1180964219:233167 (now 0 Audio frames in queue)
2007-06-04 15:36:59.260 Mischpult[12008] 1180964219:259716 -
doGetNextFrame(): provided Video frame: fFrameSize = 5938 bytes with
fPresentationTime = 1180964218:468062 (now 15 Video frames in queue)
2007-06-04 15:36:59.273 Mischpult[12008] 1180964219:266457 - buffering
video frame (now 16 video frames in queue)
2007-06-04 15:36:59.274 Mischpult[12008] 1180964219:266457 - buffering
audio frame (now 1 audio frames in queue)
2007-06-04 15:36:59.274 Mischpult[12008] 1180964219:273770 -
doGetNextFrame(): provided Audio frame: fFrameSize = 384 bytes with
fPresentationTime = 1180964219:266457 (now 0 Audio frames in queue)
2007-06-04 15:36:59.302 Mischpult[12008] 1180964219:299706 - buffering
audio frame (now 1 audio frames in queue)
2007-06-04 15:36:59.302 Mischpult[12008] 1180964219:301692 -
doGetNextFrame(): provided Audio frame: fFrameSize = 384 bytes with
fPresentationTime = 1180964219:299706 (now 0 Audio frames in queue)
2007-06-04 15:36:59.324 Mischpult[12008] 1180964219:316339 - buffering
video frame (now 17 video frames in queue)
2007-06-04 15:36:59.335 Mischpult[12008] 1180964219:332990 - buffering
audio frame (now 1 audio frames in queue)
2007-06-04 15:36:59.336 Mischpult[12008] 1180964219:335085 -
doGetNextFrame(): provided Audio frame: fFrameSize = 384 bytes with
fPresentationTime = 1180964219:332990 (now 0 Audio frames in queue)
2007-06-04 15:36:59.374 Mischpult[12008] 1180964219:366240 - buffering
video frame (now 18 video frames in queue)
2007-06-04 15:36:59.375 Mischpult[12008] 1180964219:366240 - buffering
audio frame (now 1 audio frames in queue)
2007-06-04 15:36:59.375 Mischpult[12008] 1180964219:374349 -
doGetNextFrame(): provided Audio frame: fFrameSize = 384 bytes with
fPresentationTime = 1180964219:366240 (now 0 Audio frames in queue)
2007-06-04 15:36:59.414 Mischpult[12008] 1180964219:399538 - buffering
video frame (now 19 video frames in queue)
2007-06-04 15:36:59.415 Mischpult[12008] 1180964219:399538 - buffering
audio frame (now 1 audio frames in queue)
2007-06-04 15:36:59.415 Mischpult[12008] 1180964219:414506 -
doGetNextFrame(): provided Audio frame: fFrameSize = 384 bytes with
fPresentationTime = 1180964219:399538 (now 0 Audio frames in queue)
2007-06-04 15:36:59.443 Mischpult[12008] 1180964219:432771 - buffering
audio frame (now 1 audio frames in queue)
2007-06-04 15:36:59.443 Mischpult[12008] 1180964219:442352 -
doGetNextFrame(): provided Audio frame: fFrameSize = 384 bytes with
fPresentationTime = 1180964219:432771 (now 0 Audio frames in queue)
2007-06-04 15:36:59.463 Mischpult[12008] 1180964219:449407 - buffering
video frame (now 20 video frames in queue)
2007-06-04 15:36:59.476 Mischpult[12008] 1180964219:466037 - buffering
audio frame (now 1 audio frames in queue)
2007-06-04 15:36:59.477 Mischpult[12008] 1180964219:476075 -
doGetNextFrame(): provided Audio frame: fFrameSize = 384 bytes with
fPresentationTime = 1180964219:466037 (now 0 Audio frames in queue)
2007-06-04 15:36:59.513 Mischpult[12008] 1180964219:499305 - buffering
video frame (now 21 video frames in queue)
2007-06-04 15:36:59.514 Mischpult[12008] 1180964219:499305 - buffering
audio frame (now 1 audio frames in queue)
2007-06-04 15:36:59.514 Mischpult[12008] 1180964219:513593 -
doGetNextFrame(): provided Audio frame: fFrameSize = 384 bytes with
fPresentationTime = 1180964219:499305 (now 0 Audio frames in queue)
2007-06-04 15:36:59.542 Mischpult[12008] 1180964219:532585 - buffering
audio frame (now 1 audio frames in queue)
2007-06-04 15:36:59.542 Mischpult[12008] 1180964219:541832 -
doGetNextFrame(): provided Audio frame: fFrameSize = 384 bytes with
fPresentationTime = 1180964219:532585 (now 0 Audio frames in queue)
2007-06-04 15:36:59.564 Mischpult[12008] 1180964219:549203 - buffering
video frame (now 22 video frames in queue)
2007-06-04 15:36:59.576 Mischpult[12008] 1180964219:565836 - buffering
audio frame (now 1 audio frames in queue)
2007-06-04 15:36:59.576 Mischpult[12008] 1180964219:575712 -
doGetNextFrame(): provided Audio frame: fFrameSize = 384 bytes with
fPresentationTime = 1180964219:565836 (now 0 Audio frames in queue)
2007-06-04 15:36:59.613 Mischpult[12008] 1180964219:599108 - buffering
video frame (now 23 video frames in queue)
2007-06-04 15:36:59.614 Mischpult[12008] 1180964219:599108 - buffering
audio frame (now 1 audio frames in queue)
2007-06-04 15:36:59.614 Mischpult[12008] 1180964219:613753 -
doGetNextFrame(): provided Audio frame: fFrameSize = 384 bytes with
fPresentationTime = 1180964219:599108 (now 0 Audio frames in queue)
2007-06-04 15:36:59.643 Mischpult[12008] 1180964219:632373 - buffering
audio frame (now 1 audio frames in queue)
2007-06-04 15:36:59.643 Mischpult[12008] 1180964219:642488 -
doGetNextFrame(): provided Audio frame: fFrameSize = 384 bytes with
fPresentationTime = 1180964219:632373 (now 0 Audio frames in queue)
2007-06-04 15:36:59.664 Mischpult[12008] 1180964219:649007 - buffering
video frame (now 24 video frames in queue)
2007-06-04 15:36:59.675 Mischpult[12008] 1180964219:665632 - buffering
audio frame (now 1 audio frames in queue)
2007-06-04 15:36:59.676 Mischpult[12008] 1180964219:675335 -
doGetNextFrame(): provided Audio frame: fFrameSize = 384 bytes with
fPresentationTime = 1180964219:665632 (now 0 Audio frames in queue)
2007-06-04 15:36:59.714 Mischpult[12008] 1180964219:698906 - buffering
video frame (now 25 video frames in queue)
2007-06-04 15:36:59.714 Mischpult[12008] 1180964219:698906 - buffering
audio frame (now 1 audio frames in queue)
2007-06-04 15:36:59.718 Mischpult[12008] 1180964219:717617 -
doGetNextFrame(): provided Audio frame: fFrameSize = 384 bytes with
fPresentationTime = 1180964219:698906 (now 0 Audio frames in queue)
2007-06-04 15:36:59.743 Mischpult[12008] 1180964219:732171 - buffering
audio frame (now 1 audio frames in queue)
2007-06-04 15:36:59.743 Mischpult[12008] 1180964219:742665 -
doGetNextFrame(): provided Audio frame: fFrameSize = 384 bytes with
fPresentationTime = 1180964219:732171 (now 0 Audio frames in queue)
2007-06-04 15:36:59.764 Mischpult[12008] 1180964219:748806 - buffering
video frame (now 26 video frames in queue)
2007-06-04 15:36:59.770 Mischpult[12008] 1180964219:769106 -
doGetNextFrame(): provided Video frame: fFrameSize = 4694 bytes with
fPresentationTime = 1180964218:517955 (now 25 Video frames in queue)
2007-06-04 15:36:59.775 Mischpult[12008] 1180964219:765447 - buffering
audio frame (now 1 audio frames in queue)
2007-06-04 15:36:59.776 Mischpult[12008] 1180964219:775119 -
doGetNextFrame(): provided Audio frame: fFrameSize = 384 bytes with
fPresentationTime = 1180964219:765447 (now 0 Audio frames in queue)
2007-06-04 15:36:59.813 Mischpult[12008] 1180964219:798720 - buffering
video frame (now 26 video frames in queue)
2007-06-04 15:36:59.813 Mischpult[12008] 1180964219:798720 - buffering
audio frame (now 1 audio frames in queue)
2007-06-04 15:36:59.813 Mischpult[12008] 1180964219:812770 -
doGetNextFrame(): provided Audio frame: fFrameSize = 384 bytes with
fPresentationTime = 1180964219:798720 (now 0 Audio frames in queue)
2007-06-04 15:36:59.842 Mischpult[12008] 1180964219:831974 - buffering
audio frame (now 1 audio frames in queue)
2007-06-04 15:36:59.842 Mischpult[12008] 1180964219:841810 -
doGetNextFrame(): provided Audio frame: fFrameSize = 384 bytes with
fPresentationTime = 1180964219:831974 (now 0 Audio frames in queue)
2007-06-04 15:36:59.863 Mischpult[12008] 1180964219:848610 - buffering
video frame (now 27 video frames in queue)
2007-06-04 15:36:59.875 Mischpult[12008] 1180964219:865244 - buffering
audio frame (now 1 audio frames in queue)
2007-06-04 15:36:59.879 Mischpult[12008] 1180964219:878629 -
doGetNextFrame(): provided Audio frame: fFrameSize = 384 bytes with
fPresentationTime = 1180964219:865244 (now 0 Audio frames in queue)
2007-06-04 15:36:59.913 Mischpult[12008] 1180964219:898514 - buffering
video frame (now 28 video frames in queue)
2007-06-04 15:36:59.913 Mischpult[12008] 1180964219:898514 - buffering
audio frame (now 1 audio frames in queue)
2007-06-04 15:36:59.921 Mischpult[12008] 1180964219:920452 -
doGetNextFrame(): provided Audio frame: fFrameSize = 384 bytes with
fPresentationTime = 1180964219:898514 (now 0 Audio frames in queue)
2007-06-04 15:36:59.942 Mischpult[12008] 1180964219:931772 - buffering
audio frame (now 1 audio frames in queue)
2007-06-04 15:36:59.942 Mischpult[12008] 1180964219:941443 -
doGetNextFrame(): provided Audio frame: fFrameSize = 384 bytes with
fPresentationTime = 1180964219:931772 (now 0 Audio frames in queue)
2007-06-04 15:36:59.963 Mischpult[12008] 1180964219:948405 - buffering
video frame (now 29 video frames in queue)
2007-06-04 15:36:59.975 Mischpult[12008] 1180964219:965043 - buffering
audio frame (now 1 audio frames in queue)
2007-06-04 15:36:59.976 Mischpult[12008] 1180964219:975041 -
doGetNextFrame(): provided Audio frame: fFrameSize = 384 bytes with
fPresentationTime = 1180964219:965043 (now 0 Audio frames in queue)
2007-06-04 15:37:00.013 Mischpult[12008] 1180964219:998321 - buffering
video frame (now 30 video frames in queue)
2007-06-04 15:37:00.013 Mischpult[12008] 1180964219:998321 - buffering
audio frame (now 1 audio frames in queue)
2007-06-04 15:37:00.014 Mischpult[12008] 1180964220:13239 -
doGetNextFrame(): provided Audio frame: fFrameSize = 384 bytes with
fPresentationTime = 1180964219:998321 (now 0 Audio frames in queue)
2007-06-04 15:37:00.041 Mischpult[12008] 1180964220:31567 - buffering
audio frame (now 1 audio frames in queue)
2007-06-04 15:37:00.041 Mischpult[12008] 1180964220:40842 -
doGetNextFrame(): provided Audio frame: fFrameSize = 384 bytes with
fPresentationTime = 1180964220:31567 (now 0 Audio frames in queue)
2007-06-04 15:37:00.063 Mischpult[12008] 1180964220:48199 - buffering
video frame (now 31 video frames in queue)
2007-06-04 15:37:00.075 Mischpult[12008] 1180964220:64833 - buffering
audio frame (now 1 audio frames in queue)
2007-06-04 15:37:00.078 Mischpult[12008] 1180964220:77866 -
doGetNextFrame(): provided Audio frame: fFrameSize = 384 bytes with
fPresentationTime = 1180964220:64833 (now 0 Audio frames in queue)
2007-06-04 15:37:00.113 Mischpult[12008] 1180964220:98101 - buffering
video frame (now 32 video frames in queue)
2007-06-04 15:37:00.113 Mischpult[12008] 1180964220:98101 - buffering
audio frame (now 1 audio frames in queue)
2007-06-04 15:37:00.119 Mischpult[12008] 1180964220:117985 -
doGetNextFrame(): provided Audio frame: fFrameSize = 384 bytes with
fPresentationTime = 1180964220:98101 (now 0 Audio frames in queue)
2007-06-04 15:37:00.141 Mischpult[12008] 1180964220:131378 - buffering
audio frame (now 1 audio frames in queue)
2007-06-04 15:37:00.141 Mischpult[12008] 1180964220:140775 -
doGetNextFrame(): provided Audio frame: fFrameSize = 384 bytes with
fPresentationTime = 1180964220:131378 (now 0 Audio frames in queue)
2007-06-04 15:37:00.163 Mischpult[12008] 1180964220:147998 - buffering
video frame (now 33 video frames in queue)
2007-06-04 15:37:00.175 Mischpult[12008] 1180964220:164647 - buffering
audio frame (now 1 audio frames in queue)
2007-06-04 15:37:00.175 Mischpult[12008] 1180964220:174591 -
doGetNextFrame(): provided Audio frame: fFrameSize = 384 bytes with
fPresentationTime = 1180964220:164647 (now 0 Audio frames in queue)
2007-06-04 15:37:00.212 Mischpult[12008] 1180964220:197905 - buffering
video frame (now 34 video frames in queue)
2007-06-04 15:37:00.213 Mischpult[12008] 1180964220:197905 - buffering
audio frame (now 1 audio frames in queue)
2007-06-04 15:37:00.213 Mischpult[12008] 1180964220:212492 -
doGetNextFrame(): provided Audio frame: fFrameSize = 384 bytes with
fPresentationTime = 1180964220:197905 (now 0 Audio frames in queue)
2007-06-04 15:37:00.241 Mischpult[12008] 1180964220:231172 - buffering
audio frame (now 1 audio frames in queue)
2007-06-04 15:37:00.241 Mischpult[12008] 1180964220:240791 -
doGetNextFrame(): provided Audio frame: fFrameSize = 384 bytes with
fPresentationTime = 1180964220:231172 (now 0 Audio frames in queue)
2007-06-04 15:37:00.262 Mischpult[12008] 1180964220:247802 - buffering
video frame (now 35 video frames in queue)
2007-06-04 15:37:00.269 Mischpult[12008] 1180964220:268774 -
doGetNextFrame(): provided Video frame: fFrameSize = 4600 bytes with
fPresentationTime = 1180964218:567849 (now 34 Video frames in queue)
I also added some debug printfs to MultiframedRTPSink and found out,
that the sinks are queried in the same 'asynchronous' manner (the video
sink is queried much less than the audio sink). The strange numbers in
the following are pointer values and let you distinguish the audio and
the video sink.
Beginning streaming...
Beginning to read from video input...
MultiframedRTPSink (0xef6a0c0) querying input source (0xef6a420)
MultiframedRTPSink (0xef6a0c0) querying input source (0xef6a420)
MultiframedRTPSink (0xef6a0c0) querying input source (0xef6a420)
MultiframedRTPSink (0xef6a0c0) querying input source (0xef6a420)
MultiframedRTPSink (0xef6a0c0) querying input source (0xef6a420)
Beginning to read from audio input...
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
Play this stream (from the Darwin Streaming Server) using the URL:
rtsp://127.0.0.1/test.sdp
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef6a0c0) querying input source (0xef6a420)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef6a0c0) querying input source (0xef6a420)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
MultiframedRTPSink (0xef6a0c0) querying input source (0xef6a420)
MultiframedRTPSink (0xef67490) querying input source (0xef6a710)
Now to my question: Why is the video sink and thus the video source
queried at a much lower frequency than the audio respectives?
I'm sorry for the long mail, but I hope it makes sense that way...
cheers, Severin
Ross Finlayson schrieb:
>> So I tried to use your RTSP server but it didn't change the situation.
>> My streams are still diverting. I investigated the problem further and
>> fond out, that live555 queries my sound source much more often than it
>> queries the video source. I suspect it has something to do with data
>> size
>>
>
> No, that shouldn't matter. If you (ii) give your data (audio and
> video) accurate presentation times (the "fPresentationTime" variable)
> - tied to the local 'wall clock' time (e.g., using "gettimeofday()"),
> and (ii) use RTCP (by creating "RTCPInstance" objects for each
> "RTPSink"), then audio/video sync *will* work correctly at the client
> end.
>
More information about the live-devel
mailing list