[Live-devel] question about streaming h264 video and binary data

[Ottavio Campana]

Ross Finlayson ha scritto:
>> Now I would like to stream some custom data with the video stream, thus
>> I wrote a subclass of DeviceSource that feeds the custom data to a
>> subclass of OnDemandServerMediaSubsession .
>>
>> When I try to play the stream with a "normal" player such as mplayer or
>> vlc I expect them to drop the auxiliary binary data and to display the
>> h264-encoded video. But it does not work, it seams packets are not
>> correctly received (dropped?).
> 
> If you want to stream 'custom data' that isn't in the form of H.264 NAL 
> units, then you can't include this data in the H.264 RTP packets 
> (because any receiver of these packets will expect the data to be 
> formatted according to the standard RTP payload format for H.264 
> video).  Instead, you must use a separate RTP payload format number, and 
> a separate port number, and a separate "m=" line in the SDP description.
> 
> In any case, I'm going to give support only for standard RTP payload 
> formats.

Thanks Ross for the answer, I don't want to bother you with my 
implementation, but maybe you can give me a hint about what I should 
expect from the RTP protocol.

I mean, the SDP already does what you said:

Opened URL "rtsp://192.168.0.100:8554/video_stream", returning a SDP
description:
v=0
o=- 1259156055210752 1 IN IP4 192.168.0.100
s=Video session
i=LIVE555 Streaming Media v
t=0 0
a=tool:LIVE555 Streaming Media v2009.06.02
a=type:broadcast
a=control:*
a=source-filter: incl IN IP4 * 192.168.0.100
a=rtcp-unicast: reflection
a=range:npt=0-
a=x-qt-text-nam:Video session
a=x-qt-text-inf:LIVE555 Streaming Media v
m=video 18888 RTP/AVP 96
c=IN IP4 232.83.250.127/255
a=rtpmap:96 H264/90000
a=fmtp:96
packetization-mode=1;profile-level-id=000042;sprop-parameter-sets=Z0KAHtoC0EkQ,aM48gA==
a=control:track1
m=application 18892 RTP/AVP 97
c=IN IP4 232.83.250.127/255
a=rtpmap:97 MYDATA/90000
a=control:track2

and in fact, I defined a dinamic payload such as in the case of H264 
also for the data.

I tried following the stream with wireshark, and packets are sent 
correctly every 40ms, one for H264 and one for data. The only thing that 
surprises me is that the same sequence for the number of the packets is 
used for the two streams. Thus, h264 packets have even sequence number 
and data packets have odd sequence number. Is this correct for the 
protocol? Or can this be the reason why the client does not receive the 
video stream correctly, maybe thinking that half of the packets are lost?

Ottavio