[Live-devel] RTPSource/Sink sharing the same groupsock (I think I sent this originally to the wrong group)

[Ben Rush]

Sure. And thanks, as usual, for your help. I'm sure some of the confusion
is a result of me, again, being very green. As I started typing this out I
started seeing where you're coming from, so I'll explain my situation and
then confirm that I understand what YOU'RE saying at the end.

Here is the business setting:

Imagine a hospital. There is a patient room and then a nurse room. The
nurse sits and watches video coming in from the patient rooms. When the
nurse wishes to communicate with a certain patient (she/he noticed
something from the active video feeds coming in and wants to talk to the
patient), the nurse pushes a "talk" button which then opens up an audio
channel between the patient room and nursing room. So long as the "talk"
button is active, then communication occurs much like a regular phone call
(full duplex). Patient and nurse can talk to one another, hear each other,
etc. The moment the "talk" button is unclicked by the nurse, then
everything goes mute again. Video, however, is uninterrupted during this
whole time.

What we have today is the video portion: the ability to monitor what's
going on in the rooms, and the audio portion (one way, from the patient
room to the nursing room). Both the video as well as the "one way" audio is
implemented using Live555 today. It works great. Okay? And I'm very
impressed with both the stability as well as the ease of use of Live555, so
I wish to use it further if possible.

Now, there is a lot of non-RTSP/RTP communication that goes on between the
machine in the patient room and the nurse monitoring station (network calls
to start/stop things, etc), and we're trying to cut down on that. So the
ideal would be to connect to the patient server once (as is done now for
video + one-way audio), and then simply have a "back channel" that's
automatically enabled. We'd rather not, whenever the nurse hits the "talk"
button, be forced to make more network calls to tell the machine in the
patient room to spin up another audio server. An example overview is here:
https://docs.google.com/drawings/d/1w-g0WbMy8Cm_xlbkiMzknOY3a-MU41rtk9IFx6Z2Upo/edit?usp=sharing
.

Now the reason I'm stuck on RTSP is because I have something like this
already setup and working fine using RTSP and Live555 (this executes in the
patient's room).

        ServerMediaSession* sms =
ServerMediaSession::createNew(*usageEnvironment,
            StreamingOptions::StreamName.c_str(),
StreamingOptions::StreamName.c_str(), "Live H264 Stream");

        if (StreamingOptions::DoVideo)
        {
            H264LiveServerMediaSession *liveSubSession =
H264LiveServerMediaSession::createNew(*usageEnvironment, true);
            sms->addSubsession(liveSubSession);
        }
        if (StreamingOptions::DoAudio)
        {
            WindowsAudioMediaSession* audioSession =
WindowsAudioMediaSession::createNew(*usageEnvironment, true);
            sms->addSubsession(audioSession);
        }
        rtspServer->addServerMediaSession(sms);
        char* url = rtspServer->rtspURL(sms);

I was hoping to reuse this somehow and share the "WindowsAudioMediaSession"
socket to listen to audio coming from the nurse station. Why? The machine
in the patient's room has already had an established connection with the
nurse's station and so knows the IP address, etc (individual patient rooms
may not know the IP of the nurses station and so connecting to it requires
more of these extra network calls I mentioned earlier). It seems like
bridging this two-way channel atop the existing connection simplifies
things.

So in summary the workflow is as such:

1) Nurse receives video from all rooms as is done today. There's a single
process the nurse is using that has a video player instance per room.
2) Nurse sees something interesting in the room.
3) Nurse clicks "talk" button.
4) We now initiate a two-way audio stream (nurse station connects to
patient room, this opens up a single two-way audio channel).
5) Nurse resolves the problem in the room, and unchecks the "talk" button.
Things go mute again (the patient hears nothing).

Now, I mentioned as I was typing this out I think I understood where you're
coming from. I think what you're saying is that I abandon the idea of
bridging atop the existing RTSP server (the one that, today, provides audio
+ video), and just use straight up RTP for the two-way audio channel. So
perhaps for #4, we have ANOTHER server that's JUST RTP now listening for an
incoming connection from the nurse station. And we just scrap RTSP
altogether. I can use the RTPSink/Source stuff from Live555 to build this.
Is that where you're coming from?

I hope my email was more detailed and made more sense. Thanks a bunch.


On Wed, Jun 8, 2016 at 3:42 PM Ross Finlayson <finlayson at live555.com> wrote:

> > Ultimately the goal is to have one "server" which resides in a room and
> provides video + two-way audio to a monitoring station. If the monitoring
> station wishes to have a conversation with the room, then it can do so
> using the two-way audio.
>
> From your description, it’s still not quite clear to me what you want to
> do.  In particular, what do you want to have happen to the ‘back-channel’
> audio (that would go from the receiver(s?) back to the server?  Would this
> audio just get recorded by the server somehow?  Or would it somehow get
> resent to the receiver(s) (along with the video, and any other audio)?
> Would this audio somehow get mixed with the other audio (if any) before
> getting resent?  A diagram would probably help a lot in explaining
> specifically what you want to do.
>
> Another issue is that ‘two-way media’ isn’t really part of the RTSP
> standard.  Even if you were to hack a RTSP server to accept (and do
> something, I don’t know what) with incoming ‘back-channel’ audio, then what
> about the RTSP clients?  Are there any existing RTSP clients (e.g., media
> players) out there that even take audio input (from a microphone), let
> alone try to send it back to the server?  Not as far as I know.
>
> So I suggest not using RTSP ***at all*** for this, unless you can somehow
> decouple this 'back-channel’ audio from the regular RTSP stream - e.g.,
> unless you have some sort of RTSP proxy server that ‘pulls’ (using RTSP)
> the ‘back-channel’ audio from wherever it’s supposed to come from.
>
> But first, you really need to explain better what you want to do.
>
>
> Ross Finlayson
> Live Networks, Inc.
> http://www.live555.com/
>
>
> _______________________________________________
> live-devel mailing list
> live-devel at lists.live555.com
> http://lists.live555.com/mailman/listinfo/live-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.live555.com/pipermail/live-devel/attachments/20160608/c1f367ab/attachment.html>