[Live-devel] MPEG4VideoStreamFramer and MPEG4VideoStreamDiscreteFramer questions

[Nicola Bova - Redsk]

Hi all, 

I'm building a streamer server that delivers a live MPEG4 stream
produced by a camera and encoded by a hardware encoder. 

I've already managed to get this run properly on TCP with a low latency
and now I'm going to add support to RTP through the liveMedia library. 

Following the FAQ, I have written my own "FramedSource" subclass that
encapsulates my encoder (using DeviceSource.cpp) and a subclass of 
"OnDemandServerMediaSubsession" that gets its input from the live source
using MPEG4VideoFileServerMediaSubsession as source of inspiration.
Substantially I've only changed the piece of code where the new stream
source is created:

FramedSource* MPEG4VideoCamserverServerMediaSubsession
::createNewStreamSource(unsigned /*clientSessionId*/, unsigned& estBitrate) {
  estBitrate = 500; // kbps, estimate

  // Create the video source:
  CamserverSource* camSource = CamserverSource::createNew(envir() );
  if (camSource == NULL) return NULL;

  // Create a framer for the Video Elementary Stream:
     //return MPEG4VideoStreamFramer::createNew(envir(), camSource);
     return MPEG4VideoStreamDiscreteFramer::createNew(envir(), camSource);
}

Moreover I've modified "testOnDemandRTSPServer" to use my
ServerMediaSubsession. 

Now I encounter two problems I can't solve by myself:

1. If I use MPEG4VideoStreamDiscreteFramer (as it should be, given that the
encoder produces complete frames) I get a corrupted MPEG4 stream on the
client end: the resolution is not correct (the height is wrong, the width is ok),
there are strange artifacts and so on. Mplayer complains with a lot of
output like this:

[mpeg4 @ 0xb6cc8ba8]marker missing after first P partition at 0 16
[mpeg4 @ 0xb6cc8ba8]concealing 640 DC, 640 AC, 640 MV errors
[mpeg4 @ 0xb6cc8ba8]concealing 640 DC, 640 AC, 531 MV errors

and even vlc can't play it. 
The clients don't work neither using the live rtsp://.... nor playing the stream
saved in a file by openRTSP.

However the doGetNextFrame() method on the DeviceSource is called with
right timings, without pauses (see below). 

2. If I use MPEG4VideoStreamFramer I get a correct MPEG4 stream that can
be played with no problems by Mplayer and vlc. It works both live and
by reading the file produced by openRTSP. 

However, while streaming, I get a strange behavior, no matter if I use
mplayer, vlc or openRTSP. Every few frames something (I suppose
MPEG4VideoStreamFramer) "freezes" for a few hundreds of ms (or less) so that
doGetNextFrame() is not called "in time". 
Due to this behavior the video freezes (for some ms) too... 

Here it is some debug from the DeviceSource class, SEQ is the frame
sequence number (set by the encoder) and Size the frame size.
I've inserted a blank line when the app freezes, as you can see some
frames are skipped meanwhile. 

=======================================================================
CamserverSource: SEQ = 757 - Size = 3226 - fMaxSize = 63001
CamserverSource: SEQ = 758 - Size = 15474 - fMaxSize = 59775

CamserverSource: SEQ = 760 - Size = 27523 - fMaxSize = 44301
CamserverSource: SEQ = 761 - Size = 3797 - fMaxSize = 16778
CamserverSource: SEQ = 762 - Size = 5976 - fMaxSize = 12981
CamserverSource: SEQ = 763 - Size = 11167 - fMaxSize = 7005 - TOO BIG!!!
delivering fNumTruncatedBytes = 4162, fMaxSize = 142999
CamserverSource: SEQ = 764 - Size = 9161 - fMaxSize = 138837
CamserverSource: SEQ = 765 - Size = 8403 - fMaxSize = 129676
CamserverSource: SEQ = 766 - Size = 12346 - fMaxSize = 121273
CamserverSource: SEQ = 767 - Size = 18511 - fMaxSize = 108927
CamserverSource: SEQ = 768 - Size = 19867 - fMaxSize = 90416
CamserverSource: SEQ = 769 - Size = 11677 - fMaxSize = 70549
CamserverSource: SEQ = 770 - Size = 21684 - fMaxSize = 58872
CamserverSource: SEQ = 771 - Size = 1142 - fMaxSize = 37188

CamserverSource: SEQ = 780 - Size = 22495 - fMaxSize = 36046

CamserverSource: SEQ = 790 - Size = 21674 - fMaxSize = 13551 - TOO BIG!!!
delivering fNumTruncatedBytes = 8123, fMaxSize = 136453
CamserverSource: SEQ = 791 - Size = 1133 - fMaxSize = 128330

CamserverSource: SEQ = 800 - Size = 24943 - fMaxSize = 127197
CamserverSource: SEQ = 801 - Size = 4811 - fMaxSize = 102254
CamserverSource: SEQ = 802 - Size = 3869 - fMaxSize = 97443
CamserverSource: SEQ = 803 - Size = 6208 - fMaxSize = 93574

CamserverSource: SEQ = 810 - Size = 21708 - fMaxSize = 87366
CamserverSource: SEQ = 811 - Size = 1147 - fMaxSize = 65658
CamserverSource: SEQ = 812 - Size = 1061 - fMaxSize = 64511
=======================================================================

I hope I've explained the problem clearly...
Note that if I use MPEG4VideoStreamDiscreteFramer there is no such problem.
The problem doen't seem to be related to CPU utilization or network
bandwidth. 
There is no such problem if I use the encoder with TCP (without using
liveMedia).

Maybe the problem could be related to the presentationTime?
I can't understand how fPresentationTime and fDurationInMicroseconds 
should be set, is the following code ok or I've missed something?

gettimeofday(&fPresentationTime, NULL);
fDurationInMicroseconds = 40000; //25 fps


Excuse me for the long mail and thanks
	Niki