[Live-devel] Broken data when streaming HEVC video

[Ross Finlayson]

> "checker.265" is a short video of a checker pattern moving in diagonal, this is the input video, streamed via live555MediaServer. See the screenshot "checker (MPC replay).png", it looks fine when replayed from disk. When I connect via VLC using the URL rtsp://<my-ip>/checker.265, I can see corrupted output (see screenshot "live555MediaServer to VLC.png"). When I use openRTSP, it generates the file "video-H265-1", which, when replayed is also corrupt (see screenshot "live555MediaServer to openRTSP (MPC replay).png"). 

If you look at the diagnostic output (on ‘stderr’) from the “LIVE555 Media Server”, it shows clearly what the problem is:
	MultiFramedRTPSink::afterGettingFrame1(): The input frame data was too large for our buffer size (100452). 53046 bytes of trailing data was dropped! Correct this by increasing "OutPacketBuffer::maxSize" to at least 153046, *before* creating this 'RTPSink'. (Current value is 100000.) 

In other words, your video has exceptionally large ‘key frames’; you need to increase a buffer size - in your server - to correct for this.  The place to do this is at line 146 of “mediaServer/DynamicRTSPServer.cpp”.  You should change "OutPacketBuffer::maxSize” from 100000 to 200000 (assuming that your ‘key frames’ are no larger than this).

If you do this, you will no longer see error messages output by your server.  But your excessively large key frames will still cause problems at the *client* end.  When you run “openRTSP” to receive the stream (after modifying your server), it reports:
	MultiFramedRTPSource::doGetNextFrame1(): The total received frame size exceeds the client's buffer size (100000). 53046 bytes of trailing data will be dropped! FileSink::afterGettingFrame(): The input frame data was too large for our buffer size (100000). 53046 bytes of trailing data was dropped! Correct this by increasing the "bufferSize" parameter in the "createNew()" call to at least 153046 

With “openRTSP” as your client, you can correct this simply by giving it the option “-b 200000” (see the online documentation at <http://live555.com/openRTSP/#other-options>).  If you do this, you will receive a video file that’s essentially identical to the original (except for some extra SPS,PPS,VPS NAL units that “openRTSP” writes to the start of the received video file).

With VLC as a client, however, you will also see the effects of your excessively-large key frames, but only at the start of playback.  (VLC automatically dynamically resizes its buffer when it detects that it is receiving an excessively large NAL unit (video frame).)  In other words, if you use VLC as a client (after modifying your server to increase its buffer size), you should see a glitch at the very start of playback, but not thereafter (assuming no network packet loss; see below).


But the REAL solution to your problem - as I’ve noted many times on this mailing list - is this:
----------
I recommend that you reconfigure your encoder so that each key frame is encoded as a sequence of ’slice’ NAL units, rather than as a single NAL unit.

People often have trouble streaming H.264/H.265 video with extremely large key frames, if each key frame gets encoded as a single NAL-unit.  The problem with this is that these NAL units get sent as a (very long) sequence of RTP packets - and if even one of these RTP packets gets lost, then the whole NAL (key frame in this case) will get discarded by the receiver; see
	http://lists.live555.com/pipermail/live-devel/2011-December/014190.html
	http://lists.live555.com/pipermail/live-devel/2012-August/015615.html
	http://lists.live555.com/pipermail/live-devel/2013-May/016994.html
	http://lists.live555.com/pipermail/live-devel/2014-June/018426.html
	http://lists.live555.com/pipermail/live-devel/2014-June/018432.html
	http://lists.live555.com/pipermail/live-devel/2014-June/018433.html
	http://lists.live555.com/pipermail/live-devel/2014-June/018434.html
	http://lists.live555.com/pipermail/live-devel/2015-March/019135.html
	http://lists.live555.com/pipermail/live-devel/2015-April/019228.html
For streaming, it’s better to encode large key frames as a sequence of ‘slice’ NAL units.


Ross Finlayson
Live Networks, Inc.
http://www.live555.com/