[Live-devel] Making ProxyServerMediaSubsession stream to multicast addresses

[Jan Ekholm]

On 6 apr 2014, at 22:54, Ross Finlayson <finlayson at live555.com> wrote:
> 
>> Also I probably want to also be able to decode the incoming stream in order to grab
>> still images from it and do some manipulation on it before re-encoding it again. Is there
>> some good way to "duplicate" the incoming stream into my custom code for doing the
>> image manipulation while at the same time keeping the normal proxied stream working?
> 
> Yes, you can use the "StreamReplicator" class.  (Note the "testReplicator" demo application - in "testProgs" - that illustrates how to use this.)
> 
> If you use the mechanism that I suggested above (piping "openRTSP" into a modified "testH264VideoStreamer"), then you could update the "testH264VideoStreamer" some more, by feeding the "StreamReplicator" from the input "ByteStreamFileSource" (from "stdin"), create two replicas, then feed one replica into the "H264VideoStreamFramer" (and thus the "H264VideoRTPSink", for streaming), and feed another replica into your decoder.

Finally got to the point where I'm trying to save a backup of the received stream using StreamReplicator.
It's not going too well, as expected. :)

I receive a stream from a remote camera with code based on testRTSPClient. Internally I use a RTSPClient
instance and have a very similar layout to the original code.

void RemoteRTSPClient::setupCallback () {
    ...
    // set up the sink used to multicast the stream further
    H264VideoRTPSink * sink = H264VideoRTPSink::createNew( envir(), m_rtpGroupsock, 96, m_subsession->fmtp_spropparametersets() );
    m_subsession->sink = sink;

    // create the server media session with suitable meta data
    ServerMediaSession* sms = ServerMediaSession::createNew( ... );
    sms->addSubsession( PassiveServerMediaSubsession::createNew( *sink, 0 ) );
    rtpsServer->addSession( sms );

    // set up a framer as the input consists of a sequence of discrete NAL units and not a byte stream
    H264VideoStreamDiscreteFramer * framer = H264VideoStreamDiscreteFramer::createNew( envir(), m_subsession->readSource() );

    // create a replicator to allow us to copy the stream
    m_replicator = StreamReplicator::createNew( envir(), framer, False );

    if ( ! m_subsession->sink->startPlaying( *m_replicator->createStreamReplica(), ... ) ) {
        // error
        ....
    }

    // start the stream
    sendPlayCommand( *m_session, RemoteRTSPClient::playCallback );
}

Here I only try to get the original stream to work through the replicator. Based on the testReplicator example
all streams must be through a replicator, it's not possible to have one stream use the replicator and one
stream use the normal way. That results in errors here:

void FramedSource::getNextFrame(...
  if (fIsCurrentlyAwaitingData) {
    envir() << "FramedSource[" << this << "]::getNextFrame(): attempting to read more than once at the same time!\n";
    envir().internalError();
  }

If I change it to be:

    if ( ! m_subsession->sink->startPlaying( *framer, ... ) ) {

then it works ok, but the replicator can then not be used at all due to the above check in FramedSource. 

Probably there's some reorganization that needs to be done, but I've tried quite a few permutations of
the code and nothing works. The one that almost works is:

    m_replicator = StreamReplicator::createNew( envir(), m_subsession->readSource(), False );
    H264VideoStreamDiscreteFramer * discreteFramer = H264VideoStreamDiscreteFramer::createNew( envir(), m_replicator->createStreamReplica() );
    if ( ! m_subsession->sink->startPlaying( * discreteFramer, ... ) ) {
        ....
    }

This does deliver a video stream, but it's unwatchable. The individual images are ok, but the video is stuttering
back and forth, as if it was zig-zagging over the time line. There is not too much documentation for
StreamReplicator and the provided example is quite limited. 

-- 
Jan Ekholm
jan.ekholm at d-pointer.com