[Live-devel] RTSPClient w/significant frame loss

[Brad O'Hearne]

Hello gang....

I am soldiering right along with my (now working) video processing pipeline, which constitutes pulling H.264 streaming video across the network via RTSP, and then decoding it using ffmpeg. The lion's share of my Live555 implementation resides in a RTSPClient subclass, and a MediaSink subclass, pretty much standard in relation to the testRTSPClient example out there. 

So here's my latest hurdle. I have completely unhooked the ffmpeg side of the equation, and commented out all processing that is taking place in my MediaSink's afterGettingFrame() subclass -- the only thing this method implementation is doing is logging NAL unit type counts to the console. My purpose in doing this was to determine the frame rate being received in the RTSPClient, since I control the server side of this, and know exactly what is crossing the wire (or more properly put, wireless). Here's the configuration: 

- Server is pumping out 30 frames of video per second over a closed Wifi LAN (by closed, I mean there is only one client on the network, my app, within close proximity, so there should be next to no latency). More specifically, the server is sending 1 NAL unit type 7, 1 NAL unit type 8, and 1 NAL unit type 5 (the actual data), and sending that sequence of NAL units continually. There are 30 frames a second (and when I say "frames" I am not talking all NAL units, I'm talking NAL unit type 5's, the actual video data). So with a RTSPClient that to my knowledge (as far as my subclasses are concerned) I should be getting close to 30fps. I'm not -- I'm getting around 9fps, so about 66% frame dropping. Again, my subclasses are doing zero processing, just counting NALs and occasionally logging the number to the console -- and there is no downstream decoding or processing of any kind taking place. 

My questions: 

1. What factors / culprits could be causing this massive frame loss? 

2. Is there anything in the base RTSPClient or MediaSink subclasses (or elsewhere) that can be configured to reduce frame loss? 

3. Are there any other known factors that contribute to performance that could be acting on this use case. 

Thanks for your help!

Cheers, 

Brad

Brad O'Hearne
Founder / Lead Developer
Big Hill Software LLC
http://www.bighillsoftware.com