[Live-devel] H264 Live Input Issue

[Shubham Goyal]

Hi all

Based on the FAQ for live input, I wrote my own subclasses for FramedSource and OnDemandServerMediaSubsession to implement a unicast server that can stream a live input. My live input source is a H264 encoded frame grab of my desktop using the Nvidia GRID SDK (which internally uses Nvenc.) 

Before passing the frames to live555 I also removed the start codes in every frame and created a queue of NAL units. The first frame of every stream contains 3 NALs. And a single NAL in every subsequent frame. An example of the output:

Start code found in frame 1
NAL size: 24
67 42 C0 0D 95 A0 D3 F9 67 C0 52 83 02 03 20 00 00 7D 00 00 1D 4C 10 80

Start code found in frame 1
NAL size: 4
68 CE 3C 80

Start code found in frame 1
NAL size: 3228
65 B8 04 7F C4 44 D4 08 00 0C 07 8A 00 02 01 D3 58 0E 93 93 00 2D 11 F3 D9 04 BA
 48 99 6A 65 D3 81 79 07 92 B2 95 81 1D BD CF E4 70 86 7E F1 41 B9 35 76 9C 28 2
A E6 3D 30 6A 2E C0 99 91 99 2F 22 3E FF 19 54 3B 13 4C C0 38 15 7F F2 7E FB CD
… //a lot of data, removed for readability
5B 5B 5B 5B 5F 6B FF FF C1 54 20 04 E7 38 40 25 29 41 C0 04 00 43 3C 1C 00 40 08
 67 

Start code found in frame 2 // Note: second frame here
NAL size: 1820
61 E0 20 BF C6 C1 23 80 78 54 98 07 2C C0 09 50 7A A4 C0 72 CA 91 8F 2D E5 00 4E
 82 01 04 03 A1 8B 24 A2 9C B5 F1 98 E0 3E 08 0C 0A 85 00 CF 12 05 E3 B1 36 82 F0
 … //a lot of data, removed for readability
 CC 65 72 DF FF D8 83 F3 A6 53 3F C4 2E 21 71 0B 88 5C 42 E2 17 10 BF 70 1C 43 2
A 4B 03 88 65 49 6F

//Third frame.. and so forth

Upon playing the RTSP stream using VLC my output is blank. VLC is receiving data because the seconds clock in VLC starts ticking as soon as the stream is played but the output is completely blank. I tried a lot of combinations in sending the data including the start codes, sending individual/multiple NALs per frame, but nothing is working. I was not sure if my data was correct, so I wrote it to a file (with start codes) and the file played back successfully. I even streamed the file using the “testOnDemandRTSPServer” test program and that worked too. There is definitely something that isn’t right and I’m not able to figure it out. Would appreciate if somebody could point me in the right direction. I am including some of my source code below for reference.

Thanks,
Shubham

——testOnDemandRTSPServer.cpp—— 
 //Live h264 based GRID stream
//Note: This is only a part of the file, not the entire file
  {
    char const* streamName = "grid";
    ServerMediaSession* sms
      = ServerMediaSession::createNew(*env, streamName, streamName,
				      descriptionString);
    sms->addSubsession(GridServerMediaSubsession::createNew(*env, reuseFirstSource));
    rtspServer->addServerMediaSession(sms);

    announceStream(rtspServer, sms, streamName, "no file");
  }
——testOnDemandRTSPServer.cpp—— 

——GridServerMediaSubsession.cpp——
#include "GridServerMediaSubsession.hh"
#include "H264VideoRTPSink.hh"
#include "GridSource.hh"
#include "H264VideoStreamDiscreteFramer.hh"

GridServerMediaSubsession*
GridServerMediaSubsession::createNew(UsageEnvironment& env,
					      Boolean reuseFirstSource) {
  return new GridServerMediaSubsession(env, reuseFirstSource);
}

GridServerMediaSubsession::GridServerMediaSubsession(UsageEnvironment& env, Boolean reuseFirstSource)
  : OnDemandServerMediaSubsession(env, reuseFirstSource) {
}

GridServerMediaSubsession::~GridServerMediaSubsession() {
}

FramedSource* GridServerMediaSubsession::createNewStreamSource(unsigned /*clientSessionId*/, unsigned& estBitrate) {
  estBitrate = 700; // kbps, estimate

  // Create the video source:
  GridSource* liveSource = GridSource::createNew(envir());

  // Create a framer for the Video Elementary Stream:
  return H264VideoStreamDiscreteFramer::createNew(envir(), liveSource);
}

RTPSink* GridServerMediaSubsession
::createNewRTPSink(Groupsock* rtpGroupsock,
		   unsigned char rtpPayloadTypeIfDynamic,
		   FramedSource* /*inputSource*/) {
  return H264VideoRTPSink::createNew(envir(), rtpGroupsock, rtpPayloadTypeIfDynamic);
——GridServerMediaSubsession.cpp——

—— GridSource.cpp ——

GridSource*
GridSource::createNew(UsageEnvironment& env) {
	AppArguments args; //for input configuration
	//some code removed for readability
  return new GridSource(env, args);
}

EventTriggerId GridSource::eventTriggerId = 0;

unsigned GridSource::referenceCount = 0;
unsigned GridSource::frameCount = 0;

GridSource::GridSource(UsageEnvironment& env,
			   AppArguments args)
  : FramedSource(env), fargs(args) {
  if (referenceCount == 0) {
    //encoder is initialised here. Some code removed for readability 
  }
  ++referenceCount;

  if (eventTriggerId == 0) {
    eventTriggerId = envir().taskScheduler().createEventTrigger(deliverFrame0);
  }
}

GridSource::~GridSource() {
  --referenceCount;
  if (referenceCount == 0) {
	if (encoder)
    {
        encoder->NvFBCH264Release();
    }

    // Reclaim our 'event trigger'
    envir().taskScheduler().deleteEventTrigger(eventTriggerId);
    eventTriggerId = 0;
  }
}

void GridSource::doGetNextFrame() {

	//Note: next frame is read only after previous frame’s NALs have been delivered.
	if(nalQ.isEmpty()){
	   frameCount++;
	   populateQ(); //load the NALs for each frame. NALs stored in nalQ
	}
	deliverFrame();
}

void GridSource::deliverFrame0(void* clientData) {
  ((GridSource*)clientData)->deliverFrame();
}

void GridSource::populateQ(){
//frame is grabbed by the GRID SDK
//NALs for every frame are pushed to nalQ
//Some code removed for readability

while(/*frame data is available in frame buffer*/){
    if(/*star code found in frame buffer*/)
	nalQ.pushNAL(outputPtr,byteSize); //data until next start code or until end of buffer - Note: data excludes start codes
   }

}

void GridSource::deliverFrame() {

  if (!isCurrentlyAwaitingData()) return; // we're not ready for the data yet

  NAL* nal = nalQ.popNAL();
  nal->print();

  // Deliver the data here:
  if (nal->getSize() > fMaxSize) {
    fFrameSize = fMaxSize;
    fNumTruncatedBytes = nal->getSize() - fMaxSize;
  } else {
    fFrameSize = nal->getSize();
  }
  gettimeofday(&fPresentationTime, NULL); 
  
  fDurationInMicroseconds = 1000000/30; //30 fps
  memmove(fTo, nal->getPayload(), fFrameSize);

  // After delivering the data, inform the reader that it is now available:
  FramedSource::afterGetting(this);
}

——GridSource.cpp—— 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.live555.com/pipermail/live-devel/attachments/20141223/f74450d8/attachment-0001.html>