[Live-devel] RTSPClient w/significant frame loss

[Kumar Sambhav]

Hi Brad,

I had done processing in afterGettingFrame() by copying the data to a Linux named PIPE on a ~300Mhz ARM based system. 

You can try few experiments. 

1. Did you profile them time taken for  memcpy in afterGettingFrame() ? 

2. In every afterGettingFrame() call, instead of using the new data coming , use a pre initialized  data and do processing with it. e.g a black frame 

3. You can move the processing to be done in afterGettingFrame() to a different thread. You can run live555 in one thread and ffmpeg in another and share data using PIPES or ring buffers. 
	FFMPEG may be internally using threads to decode, (not 100% sure on this)

4. Try with a low FPS / bitrate clip.

5. Just give a try on requesting TCP data from the server instead of UDP to see if the behavior changes. 


Regards,
Sambhav

On Mar 23, 2012, at 3:54 AM, Brad O'Hearne wrote:

> Ross, 
> 
> Thank you for your reply. To your comments: 
> 
> 1. I am fairly confident the increase of the receive buffer is taking, because that change alone eliminated 100% of frame loss when the afterGettingFrame() method did nothing bug log NAL counts. I was getting near 30fps. 
> 
> 2. I do not think the network has anything to do with it. The network is a closed LAN (i.e. not connected to the Internet or WAN), there are no other machines on the network (zero traffic), and the client is sitting about three feet from the Wifi router. Plus, as stated in 1, if the network were affected something, I wouldn't be seeing frame loss. 
> 
> So I eliminated all memory allocation and memcpy of the 23K of data into the new frame buffer from the receive buffer, and I was back to getting almost zero frame loss. But as soon as I added in anything else, virtually immediately returned to about 2/3 (66%) frame loss again. But now I have a good specific example to discuss -- ffmpeg decodding. Now actually, I did not add everything back in that is required in processing, just a single call to avcodec_decode_video2 -- no scaling, color conversion, or rendering was done.The processing time for this one call averaged around ~19 milliseconds, which resulted in losing 66% frame loss again. I'm starting to sense that it is immaterial what is done in afterGettingFrame(), that it is on the brink hitting its ability to process 30fps before afterGettingFrame() is even called, and no matter what is done in afterGettingFrame() is going to result in frame loss. 
> 
> I guess in light of what I've found, it makes me wonder if the fact that the testRTSPClient example doesn't actually do any processing in its afterGettingFrame() method might be more than coincidence. For comparison, I cracked open the VLC code which uses LIVE555 and found that they don't appear to go the MediaSink route at all, which makes me wonder if the general approach is flawed, and I need to go farther upstream to streamline the RTSPClient. 
> 
> Two questions: 
> 
> 1. Are you (or anyone) aware of what actual processing can realistically be done in afterGettingFrame() method without losing data? 
> 
> 2. Do you (or anybody) actually have a sample that is architected like the testRTSPClient example which actually does meaningful processing and doesn't lose major data by processing in afterGettingFrame()? 
> 
> If there's any example out there that shows it is possible to actual process video in afterGettingFrame() without major frame loss, I think that would help immensely. 
> 
> Thanks, 
> 
> Brad
> 
> Brad O'Hearne
> Founder / Lead Developer
> Big Hill Software LLC
> http://www.bighillsoftware.com
> 
> On Mar 21, 2012, at 11:09 PM, Ross Finlayson wrote:
> 
>>> 1. I followed the FAQ's recommendations, which increased the receiveBuffer to (VLC's recommended) 2000000. (Note, I also tried 400000, same results as 2000000, also note there are no truncated bytes).
>> 
>> Note also that the "increaseReceiveBufferTo()" function returns the actual resulting OS socket buffer size.  (It does this by doing a getsockopt(..., SOL_SOCKET) on the socket.)  So, if you haven't already done so, you should check the return value from these calls, just to make sure that the OS is actually setting the socket buffer size that you're requesting.  In some OS's - e.g., Linux - you have to configure the maximum possible buffer size, before you call "increaseReceiveBufferTo()".  (This is described in the FAQ entry.)
>> 
>> Apart from the OS's socket buffering, the only other possible cause for 'dropped packets' is actual packet loss occurring on the network.
>> 
>> 
>> Ross Finlayson
>> Live Networks, Inc.
>> http://www.live555.com/
>> 
>> _______________________________________________
>> live-devel mailing list
>> live-devel at lists.live555.com
>> http://lists.live555.com/mailman/listinfo/live-devel
> 
> _______________________________________________
> live-devel mailing list
> live-devel at lists.live555.com
> http://lists.live555.com/mailman/listinfo/live-devel

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.live555.com/pipermail/live-devel/attachments/20120323/1965ea32/attachment-0001.html>