[Live-devel] H.264 via RTP - ugly artifacts

Mon Feb 11 12:16:57 PST 2013

Hi,

I also collect SPS, PPS and IDR NALS prior to sending them to FFMPEG and had
forgotten about that until you mentioned it.   So I send FFMPEG buffers with
either [7,8,5] or [1] each time.  The P frames are probably distorted
because they refer to IDR reference pictures that are not correct.  I would
guess that the P frames are referring to the first [5] in your [7, 8, 5, 5]
sequence, and the last [5] in that sequence is incorrect.  Also, there is no
purpose to having more than one IDR in a row, unless the whole sequence is
IDRs (for enabling seeking to every single frame perhaps).

What is the data source for these sequences?  It does seem odd to me that an
encoder would generate these by default.

Chris Richardson

WTI

From: live-devel-bounces at ns.live555.com
[mailto:live-devel-bounces at ns.live555.com] On Behalf Of Jesse Hemingway
Sent: Monday, February 11, 2013 11:33 AM
To: LIVE555 Streaming Media - development & use
Subject: Re: [Live-devel] H.264 via RTP - ugly artifacts

Ross, I'm sorry to continue this thread on your forum, but I've gotten more
traction here than anywhere -- feel free to reject if you feel it's noise.

Jeff, to your notes: I spent a little more time experimenting.  I find that
for low-res video (e.g. 240x160) I'll just get a single IDR slice at any
time, and then decoding works as well as VLC (I think).  At higher res (e.g.
320x240) I start getting multiple IDR slices in a row, and then it's
artifact-city.  If I buffer all IDR slices before passing to
avcodec_decode_video2(), then I finally get clear-looking frames.  The
sequence looks something like: [7,8,5,5], [1], [1], [1], [1], [7,8,5,5],
[1], [1]...

So even if I solve the keyframe issue as above, the intervening P-frames
seem to be pushing pixels more than they should.  Basically at each
keyframe, I now get a clear image, and in-between, the whole scene kind of
bulges and gets more and more distorted until the next keyframe.  BTW my
test example is this one:
rtsp://media1.law.harvard.edu/Media/policy_a/2012/02/02_unger.mov (UDP
blocked, as Ross pointed out).

So two question:

A) Is it really necessary to collect all, successive I-frames to send all at
once to avcodec_decode_video2(), or might this indicate some other, larger
issue?  If I don't collect them all, only one fraction of the image is clear
at a time, with the rest of it totally blurred.

B) Why would the P-frames (sent to decoder one at a time) result in such
additive artifacts?

-Jesse

On Mon, Feb 11, 2013 at 10:10 AM, Jesse Hemingway
<Jesse.Hemingway at nerdery.com> wrote:

Thanks Jeff,

I tried out your suggestion of caching and passing the 7,8 frames before
every keyframe, so my sequence looked like
7,8,5,7,8,5,7,8,5,1,1,1,1,1,1,7,8,5,7,8,5,... however, I got exactly the
same visual artifacts.  I think I'm also safe on the prefix endian-ness, as
I pack my buffer with:

uint8_t nalStartSequence[] = { 0x00, 0x00, 0x00, 0x01 };

I'm pretty stymied, especially in light of the fact avcodec_decode_video2()
is not reporting any errors, other than the disturbingly-consistent DC, AC
and MV concealment on every frame (the error count is a constant function of
the picture dimensions). 

-Jesse

On Sun, Feb 10, 2013 at 5:29 PM, Jeff Shanab <jshanab at smartwire.com> wrote:

I had the same problem a while back. I also use live555 feeding libavcodec.
While the standard only says you need to have a 7 and an 8 before the first
5 and that after that the 5 or 1 is valid, I have had decoding trouble
because of it.
So while all the following are technically legal
7,8,5,1,1,1,1,5,1,1,1...
7,8,5,1,1,1,1,1,1......
7,8,5,5,5,1,1,1,5,1,1,1,5,1,1,1 (some axis cameras by default)

It is the decoder not live555 that needs the 7,8,5 at the beginning of every
key frame. 
I have gotten arguments about this, It seems to vary by version. The 7 and 8
packets are so small compared to 
the key frame slices [5] and the diff frames [1] that I just cache them and
inject them if missing. Not only do I get
Rock solid playback, I do not need to worry about a second user starting
late on a multicast or trouble when I seek.

BTW each and every frame needs the 0x00 0x00 0x00 0x01  (4 bytes, aka
network byte order. not a 32 bit byte that could end up with endian issues)

  _____  

 From: live-devel-bounces at ns.live555.com [live-devel-bounces at ns.live555.com]
on behalf of Jesse Hemingway [jhemingw at nerdery.com]
Sent: Friday, February 08, 2013 8:36 PM

To: LIVE555 Streaming Media - development & use

Cc: LIVE555 Streaming Media - development & use
Subject: Re: [Live-devel] H.264 via RTP - ugly artifacts

Interesting, thank you for the response.  I actually discard all but 7, 8
and 5 frames that occur before the initial 'priming' is complete, they are
just logged for completeness. So my first buffer passed is of the form [7 8
5], after which I pass all frames willy-nilly, even the timing NAL [ 6 ].
I'm wondering if my bytestream framing is wrong?  0x000001.NAL.0x00.  I read
the Appendix B bytestream syntax and also the ffmpeg NAL detection code and
it seems sufficient, but I've also read posts that indicated this simple
approach was incorrect.

<snip>

This message and any attachments contain confidential and proprietary
information, and may contain privileged information, belonging to one or
more affiliates of Windy City Wire Cable & Technology Products, LLC. No
privilege is waived by this transmission. Unauthorized use, copying or
disclosure of such information is prohibited and may be unlawful. If you
receive this message in error, please delete it from your system, destroy
any printouts or copies of it, and notify the sender immediately by e-mail
or phone.

_______________________________________________
live-devel mailing list
live-devel at lists.live555.com
http://lists.live555.com/mailman/listinfo/live-devel

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.live555.com/pipermail/live-devel/attachments/20130211/61139a80/attachment.html>