[Live-devel] Open H.264 stream frame by frame

[Joshua Kordani]

Last year when I did this, decoding h264 with libavcodec required 
finding specific nals, the sps and pps, and holding on to them for all 
subsequent decode calls.  What you'll likely end up needing to do is to 
figure out at what point in the live555 library the encoded data is 
available to be read, and then do your own handling that consists of 
looking for the sps and pps nals and then making decisions on when to 
submit the data you have to the decoder based on when you have a full 
frame to decode.  since there are a few different ways that a full frame 
can be shipped to you, you have to handle these on a case by case 
basis.  The most common one I've found is that frames are either encoded 
as one whole nal, and then for a period of frames following, subsequent 
frames are shipped as deltas, until such point as the encoder decides to 
emit a full frame again, OR explicit frame start and end markers are 
emitted, or other ways i'm sure I'm not familiar with.  Most commonly 
though is the full frame plus deltas approach for live streamed media.

You'll end up doing something like this
discard data until you read sps and pps nals from live555 (or these are 
passed to your client at stream start and are available immediately, 
usually the live555 library can tell you if this is the case)
label start:
once you see your first full frame nal, do:
sps pps frame_nal, send this in annex b format to libavcodec
then when you see delta nals,
sps pps frame_nal delta_nal, send this in annex b format to libavcodec
Append delta nals as you get them, so
sps pps frame_nal delta_nal delta_nal ... send this in annex b format to 
libavcodec
if you see a full frame nal, goto start.

after each call to libavcodec, you'll get a decoded frame, but you will 
likely have to convert it to the color format your cv library expects 
(or not, depends on what you're using)
If you do have to convert, lookup the swscale functions.

Joshua Kordani
LSA Autonomy

On 9/24/14 7:36 AM, Christiano Belli wrote:
> Hi Marcin,
>
>     Thanks for the response. Where can I get a example of use live555 
> with ffmpeg?
>
>
> Att.
>
> Christiano Belli
> Cel.:+55 (41) 8444-7468
> christiano at techideasbrasil.com <mailto:christianobelli at hotmail.com>
>
> On Wed, Sep 24, 2014 at 3:18 AM, Marcin <marcin at speed666.info 
> <mailto:marcin at speed666.info>> wrote:
>
>     Hi,
>     Really - i think that you have to decode the stream to get
>     frame-by-frame data for further analysis. Soon Ross will reply
>     also that Live555 is not responsible for decoding/encoding stream
>     but only for reciving it.
>     I think you have to use ffmpeg or libavcodec for example to decode
>     bitstream recived by live555 libs and then pass it to analyzing
>     software as RAW data.
>     Marcin
>
>     W dniu 2014-09-24 03:07, Christiano Belli pisze:
>>     Hello, I'm new with live555 and I'm so exciting in learning this.
>>     I have some questions:
>>
>>     There is a way where I can get the RTSP/H.264 stream from a ip
>>     camera and get each frame from video to do video analytics with
>>     openCV per example.
>>
>>     Does the process to get each frame from a RTSP/H.264 stream need
>>     a lot of machine resources?
>>
>>     Can I in the same time which I'm analyzing the video, save and
>>     live streaming the same stream?
>>
>>     All suggestions will help me a lot.
>>
>>     Att.
>>
>>     Christiano Belli
>>     Cel.:+55 (41) 8444-7468 <tel:%2B55%20%2841%29%208444-7468>
>>     christiano at techideasbrasil.com <mailto:christianobelli at hotmail.com>
>>
>>
>>     _______________________________________________
>>     live-devel mailing list
>>     live-devel at lists.live555.com  <mailto:live-devel at lists.live555.com>
>>     http://lists.live555.com/mailman/listinfo/live-devel
>
>
>     _______________________________________________
>     live-devel mailing list
>     live-devel at lists.live555.com <mailto:live-devel at lists.live555.com>
>     http://lists.live555.com/mailman/listinfo/live-devel
>
>
>
>
> _______________________________________________
> live-devel mailing list
> live-devel at lists.live555.com
> http://lists.live555.com/mailman/listinfo/live-devel

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.live555.com/pipermail/live-devel/attachments/20140924/dc54a07f/attachment.html>