[Live-devel] RTSP socket from a server remains open after a client disconnection

[GENESTIER Denis]

Hi,

First of all, thank you for all your quick replies.
The problem we are seeing appears on our RTSP server software installed in an environment which has a terrible quality of service (regularly the network shut down,....) and tens of front end clients connected on it. When a network shutdown appears, the clients will close their session due to a watchdog mechanism on RTP data. But they can't send or receive anything from the server, and they will try to re-establish a new whole RTSP connection. On the server side, the TCP sockets remain opened and will never be used again.

If theses sockets are not definitely closed, the limit of 1024 fid is reached after some weeks and this is due to these client disconnections. From this moment the server is not able anymore to handle new connections.
I have followed the recommendations of Warren and add a keep alive management on the RTSP socket to clean these "ghost connections" from the server after 4 minutes. It solves correctly our problem.

Ross, you can find attached the following patch I have used. It is enabled by a #define macro configuration and it only concerns the TCP socket of the RTSP connection.. What do you think about it? (N.B. : I have just done tests on a CentOS 7 distro).

Best Regards
Denis


On May 30, 2018, at 8:28 PM, Ross Finlayson <finlayson at live555.com> wrote:
> 
>> On May 30, 2018, at 7:16 PM, Warren Young <warren at etr-usa.com> wrote:
>> 
>> On May 30, 2018, at 7:34 AM, GENESTIER Denis <denis.genestier at thalesgroup.com> wrote:
>>> 
>>> Then if you disconnect the Ethernet wire of the computer B, the server will close the session, the RTP and RTCP sockets (in UDP mode), but not the RTSP connection (in TCP)
>> 
>> I’d say the real mystery here is how the UDP streams are being cut off, not why the TCP connection isn’t being dropped.
> 
> No, this isn’t a mystery at all.  The reason the UDP streams are being ‘cut off’ is because the server is closing/deleting the RTSP ‘session' (not ‘connection’) - which it does automatically after a certain period of inactivity (the parameter “reclamationTestSeconds”; by default, 65 seconds).

So there is a protocol-level keepalive in there.  I probably knew that once. :)

> If there’s a mystery, it’s why the *client* is apparently becoming inactive, but still keeping the TCP socket alive.

Denis told us: the client computer’s Ethernet cable is being unplugged, so even if you kill the software client, no RST/FIN can go out.  If the cable is left unplugged until all of the retries for that packet expire, the server will forever be under the impression that the connection is still alive.

Thus my recommendation to enable TCP keepalives.  That will detect the remote peer’s loss of access to the Ethernet media, causing Live555 to get 0 or -1 back from a recv() call, depending on how Live555’s I/O loop works.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: test.patch
Type: application/octet-stream
Size: 3637 bytes
Desc: test.patch
URL: <http://lists.live555.com/pipermail/live-devel/attachments/20180604/51267aba/attachment.obj>