[Live-devel] Broken data when streaming HEVC video

[Roland Aigner]

Hi,

I'm getting corrupted frames at the very beginning of the video when decoding HEVC streamed via FramedSource and H265VideoRTPSink. 

For testing and trying to isolate the issue, I write the raw coded and to-be-sent data to disk in my server code, and play it back, using my own decoder code, ffplay, MPC, and VLC. When I do this, there is no problem, at least none visible, also none of the players reports any errors. 

As a next step, I write the raw to-be-decoded data, as received via RTP streaming, to disk from within my client code. When I play this back, there seems to be data missing or broken. ffplay gives the following output:

Input #0, hevc, from 'test.mp4':0B f=0/0
  Duration: N/A, bitrate: N/A
    Stream #0:0: Video: hevc (Main), yuv420p(tv), 6176x2076 [SAR 1:1 DAR 1544:519], 30 fps, 30 tbr, 1200k tbn, 30 tbc
[hevc @ 0000000002ee2f60] Could not find ref with POC 1  0B f=0/0
    nan M-V:    nan fd=   0 aq=    0KB vq=  132KB sq=    0B f=0/0

This happens quite frequently, but not always. Server and client are running on the same machine for starters, so there is no network to blame. The video I am sending is quite large in terms of frame dimensions (6176x2076), so I turned the OutPacketBuffer::maxSize up to 500k, but it also happens with 3088x2076, although not as often. I googled a bit and the error "Could not find ref with POC 1" seems to hint to a demuxing problem?

In order to eliminate sources of error, I tried to reproduce this with as little of my own code as possible. For that I write a file with 6176x2076 HEVC encoded data to disk and doublecheck with ffplay -- the file seems to be fine. I use live555MediaServer.exe as an RTSP server and VLC as a client -> the video displayed by VLC is broken. I also tried to capture the received data and write it to disk via openRTSP. The produced file is also broken, as I can see when I play it back with ffplay, although there is no error reported in this case. MPC also shows broken frames, VLC refuses to render the file at all. 

So I guess, what I'm asking is: 
1) First, and most importantly: does anybody have an idea where this issue may be coming from and what I can do about it? Or how I could figure out what the actual issue is?
2) What is the best way to doublecheck in my code if the received data is indeed valid? I tried to compare the sizes of NAL packages fed in my overloaded version of FramedSource::deliverFrame vs. the package sizes I get in MediaSink::afterReading, but I cannot see any relation -- maybe this could tell something, but I'm not sure, I think I would have to become an expert in HEVC payloading first, which I'm not at this point.
3) Is there anything particular I have to do with the received data, I'm not aware of? From the MPlayer source code I got the impression, I just have to prefix the data with 0x000001 and then feed it to the decoder, which I do in my overloaded MediaSink::afterReading, but maybe I overlooked something. Anyways, this doesn't seem to be solving the issue at hand, since I can reproduce the error without using my own code at all.

Thanks,
Roland