[Live-devel] Low Frame Rate Live Streaming

Mon Nov 12 12:55:29 PST 2012

Hello,

I am using Live555 to stream raw H.264 live video via RTP/RTSP.  In order to provide the video at various frame rates, 1 or more uncompressed video samples are dropped before being passed to the H264 encoder.  This works well for the source frame rate of 29.97 fps and also frame rates of 14.985, 9.99, 5.994, and 4.995 fps.  When the frame rate is dropped further to 2.997 fps or below, video playback in VLC freezes with warning messages stating "picture is too late to be displayed (missing xxx ms)" followed by an avcodec error message "more than 5 seconds of late video -> dropping frame (computer too slow ?)".

I get this same behavior whether streaming H264 discrete NALs or AnnexB frames.  I would like to support three lower frame rates of 2.997, 1.998, and 0.999 fps but after extensive searching through the Live555 source code, the VLC source code and trying various timestamping offsets, I am unable to figure out why the problem is occuring.

Is there a correlation between video source frame rate and an allowable tolerance on the RTSP client side?  Any suggestions are greatly appreciated.

I've included part of the VLC log below, followed by the Live555 debug output:

VLC Messages

main error: ES_OUT_RESET_PCR called
main warning: picture is too late to be displayed (missing 22 ms)
main warning: picture is too late to be displayed (missing 31 ms)
main warning: picture is too late to be displayed (missing 40 ms)
main warning: picture is too late to be displayed (missing 33 ms)
main warning: picture is too late to be displayed (missing 38 ms)
avcodec error: more than 5 seconds of late video -> dropping frame (computer too slow ?)
main warning: picture is too late to be displayed (missing 4367 ms)
main warning: picture is too late to be displayed (missing 4374 ms)
main warning: picture is too late to be displayed (missing 4036 ms)
main warning: picture is too late to be displayed (missing 35 ms)
main warning: picture is too late to be displayed (missing 41 ms)
avcodec error: more than 5 seconds of late video -> dropping frame (computer too slow ?)

Live555 Debug Output

accept()ed connection from 166.201.101.178
RTSPClientConnection[13C30068]::handleRequestBytes() read 128 new bytes:OPTIONS rtsp://166.201.101.178:5544/stream RTSP/1.0
CSeq: 2
User-Agent: LibVLC/2.0.4 (LIVE555 Streaming Media v2012.09.13)

parseRTSPRequestString() succeeded, returning cmdName "OPTIONS", urlPreSuffix "", urlSuffix "stream", CSeq "2", Content-Length 0, with 0 bytes following the message.
sending response: RTSP/1.0 200 OK
CSeq: 2
Date: Mon, Nov 12 2012 20:36:12 GMT
Public: OPTIONS, DESCRIBE, SETUP, TEARDOWN, PLAY, PAUSE, GET_PARAMETER, SET_PARAMETER

RTSPClientConnection[13C30068]::handleRequestBytes() read 154 new bytes:DESCRIBE rtsp://166.201.101.178:5544/stream RTSP/1.0
CSeq: 3
User-Agent: LibVLC/2.0.4 (LIVE555 Streaming Media v2012.09.13)
Accept: application/sdp

parseRTSPRequestString() succeeded, returning cmdName "DESCRIBE", urlPreSuffix "", urlSuffix "stream", CSeq "3", Content-Length 0, with 0 bytes following the message.
sending response: RTSP/1.0 200 OK
CSeq: 3
Date: Mon, Nov 12 2012 20:36:13 GMT
Content-Base: rtsp://166.201.101.178:5544/stream/
Content-Type: application/sdp
Content-Length: 453

v=0
o=- 1352752561887715 1 IN IP4 166.201.101.178
s=Mobile-Vision
i=Live Stream
t=0 0
a=tool:LIVE555 Streaming Media v2012.10.22
a=type:broadcast
a=control:*
a=range:npt=0-
a=x-qt-text-nam:Streamer
a=x-qt-text-inf:Live Stream
m=video 0 RTP/AVP 96
c=IN IP4 0.0.0.0
b=AS:500
a=rtpmap:96 H264/90000
a=fmtp:96 packetization-mode=1;profile-level-id=42C00A;sprop-parameter-sets=Z0LACtoFB/EycBEAAAMD6QAAF3CPEiag,aM48gA==
a=control:track1
RTSPClientConnection[13C30068]::handleRequestBytes() read 185 new bytes:SETUP rtsp://166.201.101.178:5544/stream/track1 RTSP/1.0
CSeq: 4
User-Agent: LibVLC/2.0.4 (LIVE555 Streaming Media v2012.09.13)
Transport: RTP/AVP;unicast;client_port=61044-61045

parseRTSPRequestString() succeeded, returning cmdName "SETUP", urlPreSuffix "stream", urlSuffix "track1", CSeq "4", Content-Length 0, with 0 bytes following the message.
sending response: RTSP/1.0 200 OK
CSeq: 4
Date: Mon, Nov 12 2012 20:36:13 GMT
Transport: RTP/AVP;unicast;destination=166.201.101.178;source=166.201.101.178;client_port=61044-61045;server_port=6970-6971
Session: EA51CD5B

RTSPClientConnection[13C30068]::handleRequestBytes() read 164 new bytes:PLAY rtsp://166.201.101.178:5544/stream/ RTSP/1.0
CSeq: 5
User-Agent: LibVLC/2.0.4 (LIVE555 Streaming Media v2012.09.13)
Session: EA51CD5B
Range: npt=0.000-

parseRTSPRequestString() succeeded, returning cmdName "PLAY", urlPreSuffix "stream", urlSuffix "", CSeq "5", Content-Length 0, with 0 bytes following the message.
RTCPInstance[13C80A58]::RTCPInstance()
schedule(2.635291->1352752575.699293)
sending response: RTSP/1.0 200 OK
CSeq: 5
Date: Mon, Nov 12 2012 20:36:13 GMT
Range: npt=0.000-
Session: EA51CD5B
RTP-Info: url=rtsp://166.201.101.178:5544/stream/track1;seq=34988;rtptime=1296858229

RTSPClientConnection[13C30068]::handleRequestBytes() read 154 new bytes:GET_PARAMETER rtsp://166.201.101.178:5544/stream/ RTSP/1.0
CSeq: 6
User-Agent: LibVLC/2.0.4 (LIVE555 Streaming Media v2012.09.13)
Session: EA51CD5B

parseRTSPRequestString() succeeded, returning cmdName "GET_PARAMETER", urlPreSuffix "stream", urlSuffix "", CSeq "6", Content-Length 0, with 0 bytes following the message
.
sending response: RTSP/1.0 200 OK
CSeq: 6
Date: Mon, Nov 12 2012 20:36:13 GMT
Session: EA51CD5B

[13C80A58]saw incoming RTCP packet (from address 166.201.101.178, port 61045)
81c90007 e19e90c2 6a65ecb8 00ffffff 00018900 0000006e 00000000 00000000 81ca0004 e19e90c2 01094441 56454d2d 44543200
RR
RTSP client session (id "EA51CD5B", stream name "stream"): Liveness indication
validated RTCP subpacket (type 2): 1, 201, 0, 0xe19e90c2
UNSUPPORTED TYPE(0xca)
validated RTCP subpacket (type 2): 1, 202, 12, 0xe19e90c2
validated entire RTCP packet
sending REPORT
sending RTCP packet
80c80006 6a65ecb8 d44bdc3f b4101f32 e79b102c 00000063 0001f667 81ca0004 6a65ecb8 01094441 56454d2d 44543200
schedule(1.521657->1352752577.231249)
schedule(1.073723->1352752578.305810)
schedule(1.315771->1352752579.622907)
sending REPORT
sending RTCP packet
80c80006 6a65ecb8 d44bdc43 a0232096 e7a0730f 00000120 0005b484 81ca0004 6a65ecb8 01094441 56454d2d 44543200
schedule(2.467409->1352752582.098886)
[13C80A58]saw incoming RTCP packet (from address 166.201.101.178, port 61045)
81c90007 e19e90c2 6a65ecb8 00ffffff 00018a15 00000155 dc43a023 00010f4d 81ca0004 e19e90c2 01094441 56454d2d 44543200
RR
RTSP client session (id "EA51CD5B", stream name "stream"): Liveness indication
validated RTCP subpacket (type 2): 1, 201, 0, 0xe19e90c2
UNSUPPORTED TYPE(0xca)
validated RTCP subpacket (type 2): 1, 202, 12, 0xe19e90c2
validated entire RTCP packet
sending REPORT
sending RTCP packet
80c80006 6a65ecb8 d44bdc46 1ac4e725 e7a3da98 000001a5 00085e35 81ca0004 6a65ecb8 01094441 56454d2d 44543200
schedule(4.129500->1352752586.241355)
[13C80A58]saw incoming RTCP packet (from address 166.201.101.178, port 61045)
81c90007 e19e90c2 6a65ecb8 00ffffff 00018b27 000001f5 dc461ac4 0003ecb4 81ca0004 e19e90c2 01094441 56454d2d 44543200
RR
RTSP client session (id "EA51CD5B", stream name "stream"): Liveness indication
validated RTCP subpacket (type 2): 1, 201, 0, 0xe19e90c2
UNSUPPORTED TYPE(0xca)
validated RTCP subpacket (type 2): 1, 202, 12, 0xe19e90c2
validated entire RTCP packet
schedule(0.379229->1352752586.621720)
schedule(0.755303->1352752587.377771)
sending REPORT
sending RTCP packet
80c80006 6a65ecb8 d44bdc4b 60fd4bf1 e7ab18d7 000002af 000da6ca 81ca0004 6a65ecb8 01094441 56454d2d 44543200
reap: checking SSRC 0xe19e90c2: 4 (threshold 0)
schedule(4.851568->1352752592.235369)
RTSPClientConnection[13C30068]::handleRequestBytes() read 149 new bytes:TEARDOWN rtsp://166.201.101.178:5544/stream/ RTSP/1.0
CSeq: 7
User-Agent: LibVLC/2.0.4 (LIVE555 Streaming Media v2012.09.13)
Session: EA51CD5B

parseRTSPRequestString() succeeded, returning cmdName "TEARDOWN", urlPreSuffix "stream", urlSuffix "", CSeq "7", Content-Length 0, with 0 bytes following the message.
RTCPInstance[13C80A58]::~RTCPInstance()
sending BYE
sending RTCP packet
80c80006 6a65ecb8 d44bdc4f 9dd9c27f e7b0eaab 00000373 001183b5 81cb0001 6a65ecb8
sending response: RTSP/1.0 200 OK
CSeq: 7
Date: Mon, Nov 12 2012 20:36:31 GMT

RTSPClientConnection[13C30068]::handleRequestBytes() read -1 new bytes (of 10000); terminating connection!

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.live555.com/pipermail/live-devel/attachments/20121112/d2742a54/attachment-0001.html>