[Live-devel] Need help: RTSP Stream -> video render
Jon Burgess
jkburges at gmail.com
Mon Feb 27 14:37:08 PST 2012
Hi Brad,
> What I need to do is take the data received over RTSP, decode the H.264
> video, and then output it to the screen. It would seem that this wouldn't
> be too utterly terrible. However, referencing some of these libraries /
> headers inside Xcode, and trying to move some of this code around into a
> more Objective-C friendly fashion is giving me fits.
>
> If there is anyone out there familiar with using Live555 on iOS, or anyone
> who can give guidance here, I would very much appreciate it. Please feel
> free to reply here, or contact me offline as well at
> brado at bighillsoftware.com.
>
>
It's a bit of a vague question, so I'll give you some vague advice. If you
want live555 and cocoa/iOS to play nicely with each other, I'd suggest
providing a cocoa based implementation of the TaskScheduler (and also
potentially the UsageEnvironment), so that live555 events are posted and
consumed via the cocoa event loop. See
http://www.live555.com/liveMedia/faq.html#threads for more info on what
those classes are for. Basically this means you can use live555 on the
same thread as the main iOS one, and potentially avoid a lot of threading
issues.
I've posted my implementation at the bottom of this email, feel free to
use, although no guarantee can be made as to the quality! Regarding
rendering of data, for my app, I'm rendering audio rather than video. I've
subclassed MediaSink, with the implementation taking packets of data from
live555 and sending to the iOS audio queue. You will want to do a similar
thing, except with an extra decoding step in there and sending to some sort
of video/image buffer (not familiar with what's available on iOS, although
I would've thought that there'd be an API to decode H.264 in the iOS API so
you may not need ffmpeg).
Cheers,
Jon
/*
* CocoaTaskScheduler.h
* openRTSP
*
* Created by Jon Burgess on 22/10/10.
* Copyright Jon Burgess. All rights reserved.
*
*/
#ifndef COCOA_TASK_SCHEDULER_HH
#define COCOA_TASK_SCHEDULER_HH
#include "BasicUsageEnvironment.hh"
#include <map>
class CocoaTaskScheduler : public BasicTaskScheduler
{
public:
CocoaTaskScheduler(); // abstract base class
virtual ~CocoaTaskScheduler();
virtual TaskToken scheduleDelayedTask(int64_t microseconds, TaskFunc*
proc,
void* clientData);
// Schedules a task to occur (after a delay) when we next
// reach a scheduling point.
// (Does not delay if "microseconds" <= 0)
// Returns a token that can be used in a subsequent call to
// unscheduleDelayedTask()
virtual void unscheduleDelayedTask(TaskToken& prevTask);
// (Has no effect if "prevTask" == NULL)
// Sets "prevTask" to NULL afterwards.
virtual void rescheduleDelayedTask(TaskToken& task,
int64_t microseconds, TaskFunc* proc,
void* clientData);
// Combines "unscheduleDelayedTask()" with "scheduleDelayedTask()"
// (setting "task" to the new task token).
// For handling socket operations in the background (from the event
loop):
// typedef void BackgroundHandlerProc(void* clientData, int mask);
// Possible bits to set in "mask". (These are deliberately defined
// the same as those in Tcl, to make a Tcl-based subclass easy.)
virtual void setBackgroundHandling(int socketNum, int conditionSet,
BackgroundHandlerProc* handlerProc, void* clientData);
virtual void moveSocketHandling(int oldSocketNum, int newSocketNum);
// Changes any socket handling for "oldSocketNum" so that occurs with
"newSocketNum" instead.
virtual void doEventLoop(char* watchVariable = NULL);
// Stops the current thread of control from proceeding,
// but allows delayed tasks (and/or background I/O handling)
// to proceed.
// (If "watchVariable" is not NULL, then we return from this
// routine when *watchVariable != 0)
private:
/**
* This acts as a bridge between the NSInvocation given to an NSTimer
* and the TaskFunc function pointer.
*/
void* mTimerInvokee;
/**
* Keep a map of socket to run loop ref, so that we can remove socket
handling
* from run loop.
*/
// TODO: shouldn't be a void* - compilation problems though.
std::map<int, void*> mSocketToRunLoopRef;
/**
* Map of socket to CFSocketRef (for later invalidation).
* TODO:
*/
std::map<int, void*> mSocketToCFSocket;
};
#endif /* COCOA_TASK_SCHEDULER_HH */
/*
* CocoaTaskScheduler.cpp
* openRTSP
*
* Created by Jon Burgess on 22/10/10.
* Copyright Jon Burgess. All rights reserved.
*
*/
#include "CocoaTaskScheduler.h"
#import <Foundation/Foundation.h>
#include <assert.h>
#include "constants.h"
#import "TimerInvokee.h"
CocoaTaskScheduler::CocoaTaskScheduler()
: BasicTaskScheduler()
{
mTimerInvokee = [[TimerInvokee alloc] init];
}
CocoaTaskScheduler::~CocoaTaskScheduler()
{
[(TimerInvokee*)mTimerInvokee release];
}
// Schedules a task to occur (after a delay) when we next
// reach a scheduling point.
// (Does not delay if "microseconds" <= 0)
// Returns a token that can be used in a subsequent call to
// unscheduleDelayedTask()
TaskToken CocoaTaskScheduler::scheduleDelayedTask(int64_t microseconds,
TaskFunc* proc,
void* clientData)
{
NSTimeInterval seconds = microseconds / MICROS_IN_SEC;
SEL timerSelector = @selector(execute:clientData:invoker:);
NSMethodSignature* methodSig = [(TimerInvokee*)mTimerInvokee
methodSignatureForSelector:timerSelector];
NSInvocation* invocation = [NSInvocation
invocationWithMethodSignature:methodSig];
[invocation setTarget:(TimerInvokee*)mTimerInvokee];
[invocation setSelector:timerSelector];
[invocation setArgument:&proc atIndex:2];
[invocation setArgument:&clientData atIndex:3];
NSTimer* timer = [NSTimer scheduledTimerWithTimeInterval:seconds
invocation:invocation
repeats:NO];
[invocation setArgument:&timer atIndex:4];
// Need an extra retain here (maybe) because otherwise "unschedule...()"
// may be called with an invalid NSTimer.
[timer retain];
// NSLog(@"Scheduled task: %u, microseconds: %li, proc: %i", timer,
microseconds, proc);
return timer;
}
// (Has no effect if "prevTask" == NULL)
// Sets "prevTask" to NULL afterwards.
void CocoaTaskScheduler::unscheduleDelayedTask(TaskToken& prevTask)
{
if (prevTask == NULL)
{
return;
}
NSTimer* timer = (NSTimer*)prevTask;
if (timer == nil)
{
// Do nothing.
}
else if (![timer isValid])
{
NSLog(@"unscheduleDelayedTask: invalid timer, nothing to do.");
}
else
{
// NSLog(@"Unscheduling task: %u", timer);
[timer invalidate];
timer = nil;
// NSLog(@"Unscheduled task: %u", timer);
}
//[timer release];
prevTask = NULL;
}
// Combines "unscheduleDelayedTask()" with "scheduleDelayedTask()"
// (setting "task" to the new task token).
void CocoaTaskScheduler::rescheduleDelayedTask(TaskToken& task,
int64_t microseconds, TaskFunc* proc,
void* clientData)
{
// NSLog(@"Reschedule task: %u", task);
unscheduleDelayedTask(task);
task = scheduleDelayedTask(microseconds, proc, clientData);
}
struct HandlerAndClientData
{
CocoaTaskScheduler::BackgroundHandlerProc* handlerProc;
void* clientData;
};
typedef struct HandlerAndClientData HandlerAndClientData;
static void MyCallBack (
CFSocketRef s,
CFSocketCallBackType callbackType,
CFDataRef address,
const void *data,
void *info
)
{
HandlerAndClientData* handlerAndClientData =
(HandlerAndClientData*)info;
CocoaTaskScheduler::BackgroundHandlerProc* handlerProc =
handlerAndClientData->handlerProc;
void* clientData = handlerAndClientData->clientData;
int resultConditionSet = 0;
if (callbackType == kCFSocketReadCallBack)
{
// NSLog(@"Socket read callback for socket ref: %u", s);
resultConditionSet |= SOCKET_READABLE;
}
if (callbackType == kCFSocketWriteCallBack)
{
// NSLog(@"Socket write callback for socket ref: %u", s);
resultConditionSet |= SOCKET_WRITABLE;
}
// TODO: SOCKET_EXCEPTION
(*handlerProc)(clientData, resultConditionSet);
}
void CocoaTaskScheduler::setBackgroundHandling(int socketNum,
int conditionSet,
BackgroundHandlerProc*
handlerProc,
void* clientData)
{
// NSLog(@"CocoaTaskScheduler::setBackgroundHandling(), socketNum: %i,
condition set: %i, handler: %u",
// socketNum, conditionSet, handlerProc);
//
// NSLog(@"Start of setBackgroundHandling");
// std::map<int, void*>::iterator iter;
// for (iter = mSocketToRunLoopRef.begin();
// iter != mSocketToRunLoopRef.end();
// ++iter)
// {
// NSLog(@"socketNum: %i", iter->first);
// }
// NSLog(@"*****");
if ( (conditionSet == 0)
&& (mSocketToRunLoopRef.find(socketNum) !=
mSocketToRunLoopRef.end()))
{
// Remove socket handling.
CFRunLoopSourceRef runLoopSourceRef =
(CFRunLoopSourceRef)mSocketToRunLoopRef[socketNum];
assert(runLoopSourceRef != NULL);
CFRunLoopRemoveSource(CFRunLoopGetCurrent(), runLoopSourceRef,
kCFRunLoopDefaultMode);
// Remove from the map.
mSocketToRunLoopRef.erase(socketNum);
// Invalidate the CFSocket (so that the same native socket can be
used
// again in a different CFSocket).
// TODO: revisit, probably very inefficient to not re-use CFSockets.
CFSocketRef socketToRemove =
(CFSocketRef)mSocketToCFSocket[socketNum];
assert(socketToRemove != NULL);
CFSocketInvalidate(socketToRemove);
mSocketToCFSocket.erase(socketNum);
// NSLog(@"Removed background handling for socket %i", socketNum);
}
else
{
// If the socket is already being used as a run loop source, clean
up
// before trying to register it again (or the new handler won't be
used.
if (mSocketToRunLoopRef.find(socketNum) !=
mSocketToRunLoopRef.end())
{
// NSLog(@"Socket %i already used, removing from run loop before
re-adding...",
// socketNum);
setBackgroundHandling(socketNum,
0,
NULL, // handlerProc
NULL); // clientData
}
CFOptionFlags optionFlags = kCFSocketNoCallBack;
if (conditionSet & SOCKET_READABLE)
{
optionFlags |= kCFSocketReadCallBack;
// NSLog(@"Enabled read callback for socket %i", socketNum);
}
if (conditionSet & SOCKET_WRITABLE)
{
optionFlags |= kCFSocketWriteCallBack;
// NSLog(@"Enabled write callback for socket %i", socketNum);
}
if (conditionSet & SOCKET_EXCEPTION)
{
// optionFlags |= TODO
}
CFSocketCallBack callback = MyCallBack; // TODO
HandlerAndClientData* handlerAndClientData =
(HandlerAndClientData*)malloc(sizeof(HandlerAndClientData));
handlerAndClientData->handlerProc = handlerProc;
handlerAndClientData->clientData = clientData;
// TODO: leak.
CFSocketContext* pSocketContext =
(CFSocketContext*)malloc(sizeof(CFSocketContext));
pSocketContext->version = 0;
pSocketContext->info = handlerAndClientData;
pSocketContext->retain = NULL; // TODO: revisit this.
pSocketContext->release = NULL;
pSocketContext->copyDescription = NULL;
CFSocketRef socketRef = CFSocketCreateWithNative(NULL, socketNum,
optionFlags, callback, pSocketContext);
mSocketToCFSocket[socketNum] = socketRef;
NSLog(@"Created CFSocket %u for socketNum %i", socketRef,
socketNum);
// Don't close the native socket when the CFSocket is invalidated.
optionFlags = CFSocketGetSocketFlags(socketRef);
optionFlags &= ~kCFSocketCloseOnInvalidate;
CFSocketSetSocketFlags(socketRef, optionFlags);
CFRunLoopSourceRef runLoopSourceRef =
CFSocketCreateRunLoopSource(NULL,
socketRef,
0);
CFRunLoopAddSource(CFRunLoopGetCurrent(), runLoopSourceRef,
kCFRunLoopDefaultMode);
// Record the mapping between socket and run loop source, so that
we can
// remove it later.
mSocketToRunLoopRef[socketNum] = runLoopSourceRef;
}
// NSLog(@"End of setBackgroundHandling");
// for (iter = mSocketToRunLoopRef.begin();
// iter != mSocketToRunLoopRef.end();
// ++iter)
// {
// NSLog(@"socketNum: %i", iter->first);
// }
// NSLog(@"*****");
}
void CocoaTaskScheduler::moveSocketHandling(int oldSocketNum, int
newSocketNum)
{
assert(false);
}
// Changes any socket handling for "oldSocketNum" so that occurs with
"newSocketNum" instead.
// Stops the current thread of control from proceeding,
// but allows delayed tasks (and/or background I/O handling)
// to proceed.
// (If "watchVariable" is not NULL, then we return from this
// routine when *watchVariable != 0)
//
void CocoaTaskScheduler::doEventLoop(char* watchVariable)
{
// Don't think it is necessary to do anything here,
// as the iOS main event loop will be used.
NSLog(@"CocoaTaskScheduler::doEventLoop() called");
}
/**
// TODO: maybe we do need to handle watchVariable though?
if ((watchVariable != NULL) && (*watchVariable != 0))
{
// NSAssert(false, "watchVariable is set");
assert(false);
}
}
*/
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.live555.com/pipermail/live-devel/attachments/20120228/5efb61a8/attachment-0001.html>
More information about the live-devel
mailing list