[Live-devel] High bit-rate (CBR) 4K streaming over a wireless

[Warren Young]

On Aug 28, 2018, at 4:25 PM, Shyam Kaundinya <shyam.kaundinya at digitalforcetech.com> wrote:
> 
> At times I see artifacts on the Rx side. Any thoughts on what may cause this behavior and how to fix it ?

Live555 is set up as a real-time streaming library primarily, not a YouTube style buffer-and-play pseudo-streaming system.  That has several implications:

1. If you’re using UDP, either because that’s what an RTSP client negotiates or because you’re using multicast, then if you lose a single packet, you’re going to corrupt the video stream for an average of GOPsize/fps/2.  If the GOP size is 300 frames and you’re streaming at 30 fps, that means your average error run length for a lost packet is 5 seconds.  (That’s a typical GOP length for highly-compressed H.264 intended for HTTP pseudo-streaming.)

2. If you’re using RTP-over-TCP, RTP-over-HTTP, or similar, then it’s *possible* the TCP stack’s retransmissions in the above case will get the lost/delayed packet to the receiver in time, but realize that at 30 fps, it only has about 33 ms to pull that trick off before the decoder starts throwing errors due to not having the data it needs.

3. WiFi speed ratings are bogus in practice.  To get the rated speed, a) your receiving antenna has to be a millimeter from the transmitter’s antenna, b) there must be no interference, c) there must be no reflections or signal grounding, and d) there can be no other clients on the network:

3a. Like any other radiated energy, WiFi power falls off as the square of the distance.  Since that directly affects SNR, that means bandwidth falls off at roughly the same rate.  Double the distance, halve the bandwidth.  If you’re 10m away from the antenna, you can expect to get about twice the data throughput as at 20m.  There’s a limit to this, where SNR is sufficient at that or any closer distance, but you can exceed this limit inside a single room.

3b. WiFi uses junk spectrum, ceded by the first big grabbers of spectrum because it’s the frequency where water molecules vibrate, which means any moisture in the air soaks up WiFi energy, thus dropping bandwidth.  That’s why microwave ovens also run on 2.4 GHz!  Not only must you contend with the humidity in the air, there’s also all of those inconvenient ugly bags of mostly water running around.[1]

3c. Many buildings are made of wifi-hostile materials such as cinderblock and rebar, which blocks much of the wifi energy at each wall transition.  Some of the energy also reflects back to the antenna, which the transmitter must cope with, at some expense to performance.

3d. WiFi is a shared medium: with a perfectly fair implementation, 10 users at 10 Mbps will saturate a 100 Mbit/sec wifi network.  WiFi implementations aren’t completely fair, so when little Joey opens YouTube on his iPad, it’ll likely break your 4K video stream.  Ask me how I know. :)

So, how to fix it?

First option: Don’t use WiFi for high-bandwidth video.  There’s a reason broadcast-quality video decoders still have Ethernet ports and don’t even bother with wifi ports, even in 2018.

Second option: If you absolutely must do this, take a cue from YouTube: switch to a buffer-and-playback model.  You must have those several seconds of buffer to recover from problems 1 thru 3d!

[1]: https://en.wikipedia.org/wiki/Home_Soil

> 7/	What changes if any would I need to make to also be able to support VBR in the same scenario - high bit rate streams over a wireless link - with live proxy servers on both Tx and Rx sides ?


The trick is to send one frame every 33 ms for 30 fps video, regardless of how large it is.  That means you have to know the upcoming size for every frame, and you have to be able to send differing size frames in constant time.

This is an excellent use for control theory:

    https://en.wikipedia.org/wiki/Control_theory

For example, a carefully tuned PID controller might be able to keep the frame rate constant in the face of wildly varying data rates.

Yes, “wildly.”  An H.264 B frame is far, far smaller than the I frame it [indirectly] refers to, yet the I frame must be sent in the same amount of time as the B frame.  At least, within the limits of the downstream decoder’s buffering abilities.  

This is called VBV in MPEG-2 contexts.  I don’t know if the ISO Media/MPEG-4/H.264 world has another term for it.

All of this is complicated, which is why Live555 currently restricts the input stream to CBR, or at least to near-CBR.

I’d love for someone to implement such a VBR-tolerant streaming algorithm.  I could do it, but decided it wasn’t worth my time relative to just working out how to null-stuff VBR streams to CBR.