[Live-devel] RTSPClient w/significant frame loss

[Brad O'Hearne]

Ross, 

I am attempting to make sense of what I am seeing, and this statement: 

> This is getting silly.  You're doing a lot of mental 'flailing around'; yet the answer is right in front of you:

is not necessarily accurate. Up until this most recent example I provided, I reported 66% frame loss without even doing any decoding attempts at all, but instead a simple memcpy of 23K to create a new frame with the received data. For further interest, I also took 100% of the memcpy (afterGettingFrame()) work and dropped it into a block that I sent to a dispatch_async call to Grand Central Dispatch which immediately returned. It changed nothing. In other words, I cannot even dispatch any work asynchronously without suffering 2/3 frame loss. 

Your statement, 

> 	while (1) {
> 		Step A: Get a frame of data from a socket;
> 		Step B: Do something with this frame of data;
> 	}

Doesn't necessarily pin the issue on Step B. It pins the issue on Step A OR Step B, or how both steps do things together based on their design. Let's do the math. In the example below, I'm getting around 10fps where 19ms of that is decoding. That means that every second, step B is 190ms. It also means that Step A is 810ms. Now I'm not saying that LIVE555 is inefficient -- not saying that at all. All I'm saying is this: 

- There's something else very curious about the situations I've encountered where there is frame loss vs. when there isn't. When I'm only logging frame counts, i.e. doing nothing in the afterGettingFrame() method, I can get 30fps. That means that in this case, LIVE555 (Step A) is consuming a maximum of ~33ms per frame in order to achieve that frame rate. However, in the case of adding the one decode call, which dropped frame rate down to 10fps, LIVE555 (Step A) is consuming ~810ms total, or 81ms processing time per frame, because the decode consumes only 19ms. That is a 41% jump in processing time for LIVE555 (Step A) when a situation is introduced where frames cannot be processed fast enough to keep from losing frames. That would seem to suggest that as soon as frame loss occurs, LIVE555 incurs additional processing, which of course would compound the problem. Perhaps there's another explanation, but it doesn't make sense -- I would expect LIVE555's overhead to be fixed per frame. 

- Bottom line, all I'm doing is inquiring as to whether there might be possibilities to optimize things further upstream. I think that question is especially justified given that VLC, whom I looked at in lieu of not having any other example of a known working model, doesn't follow the testRTSPClient example approach. 

- I do not see any working example published that actually does something meaningful in its processing. The present testRTSPClient example app just outputs to a log. I can do that fine and get 30fps. Is there a known example out there that is architected this way, does decoding and does get 30fps (in other words, a video player example)? I'm not saying it can't, I'm just saying that I'd like to see it and run it myself before I throw my hands up in the air, and write off the whole thing as impossible. 

In conclusion, I'm not sure what the source of frustration is, but please don't take anything personally and see these questions for what they are. I'm just reporting the numbers I see, and asking questions based on them. And that would appear to be the proper protocol, as there's not much documentation either within the code or supplemental to help out here, so the only real resource is this mailing list. I also presume this is the desired scenario, as when I posted about possibly documenting out this use case, it never received a reply. 

FWIW, I'm not the only one in this boat, and this use case is not uncommon. I know, because since I started posting on this mailing list several weeks ago, I've been contacted offline several times by folks that have exactly the same problem they are trying to solve and aren't making much headway with the resources available. So please, don't take offense or check out...this is how problems get solved, people get educated, and code gets better. 

Cheers, 

Brad

Brad O'Hearne
Founder / Lead Developer
Big Hill Software LLC
http://www.bighillsoftware.com

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.live555.com/pipermail/live-devel/attachments/20120323/4450ffe1/attachment-0001.html>