[Live-devel] Segmentation fault on streaming server with several unicast clients

[Bruno Abreu]

Hello Ross,

in http://lists.live555.com/pipermail/live-devel/2013-January/016347.html I
described the following scenario: "We have been running for some time a
RTSPServer that acquires video from a live source (with our own DeviceSource)
from which we create 2 or 3 replicas: one for multicast streaming and another
to feed a FileSink for file storage. The third replica is created on demand
for unicast streaming."

Our stream consists in a H264-ES video which we encapsulate in MPEG2-TS.

This is still the case today.

Currently we've been facing a segmentation fault with some of our servers. We
still haven't been able to identify the problem but we believe it might be
related to the use of the live555 libraries. So, we were hoping you, or
someone on the mailing list, could give us some pointers and, maybe, shed some
light on this matter.

Here's current the scenario:
- servers are streaming (both unicast and multicast) and recording live from 4
different cameras
- video source from cameras is H264-ES which is then packed in a MPEG2-TS
container for streaming
- this is what our streaming  pipeline looks like:
  DeviceSource -> StreamReplicator -> H264VideoStreamDiscreteFramer ->
MPEG2TransportStreamFromESSource -> SimpleRTPSink
- crash only occurs on servers that are constantly creating unicast sessions
- the more frequent the number of unicast requests the more frequently the
crash occurs
- in every server, the last log message output before the crash informs about
an RTSPClientSession (our own extension) being destroyed
- from the core dumps it's hard to tell which action causes the crash since
it's an asynchronous event but the stack trace is always the same:
«
(gdb) bt
#0  0x636e456c in ?? ()
#1  0x08091c89 in FramedSource::afterGetting (source=0x8647d50) at
FramedSource.cpp:91
#2  0x080d8825 in AlarmHandler::handleTimeout (this=0x85d74b8) at
BasicTaskScheduler0.cpp:34
#3  0x080d7476 in DelayQueue::handleAlarm (this=0x85be714) at
DelayQueue.cpp:187
#4  0x080d6aad in BasicTaskScheduler::SingleStep (this=0x85be710,
maxDelayTime=0) at BasicTaskScheduler.cpp:212
#5  0x080d7f89 in BasicTaskScheduler0::doEventLoop (this=0x85be710,
watchVariable=0xbfbc96e3 "") at BasicTaskScheduler0.cpp:80
#6  0x0805718b in main (argc=9, argv=0xbfbc9904) at smartCodecServer.cpp:569
»
- different versions of live555 have different behaviors:
  - we have the problem with the latest version (live.2016.02.22)
  - with version live.2013.01.25 (pretty old, we know) the problem did not happen

Typical client behavior (the one that creates more unicast sessions):
- connects to servers and requests several unicast streams simultaneously
- sessions don't last long and it's number is continuously changing
- most of them aren't well behaved (do not send TEARDOWN)

So far we haven't been able to reproduce the problem in our lab.

We are also attaching the output of the RTSPServer compiled with DEBUG flag.

What additional info should we provide?
What should we be looking for?
Any help will be appreciated.

Thank You,
Bruno Abreu

-- 
Living Data - Sistemas de Informação e Apoio à Decisão, Lda.

LxFactory - Rua Rodrigues de Faria,   Mobile: +351 963428802
103, edifício I - 4º piso             Phone:  +351 213622163
1300-501 LISBOA                       Fax:    +351 213622165
Portugal                              URL: www.livingdata.pt
-------------- next part --------------
A non-text attachment was scrubbed...
Name: rtsp.log.gz
Type: application/gzip
Size: 32174 bytes
Desc: not available
URL: <http://lists.live555.com/pipermail/live-devel/attachments/20160314/6d5fed35/attachment.bin>