[Live-devel] Urgent help needed!

[Morgan Tørvolt]

> Please let us know if you have any specific suggestions for
> improvements to the existing "MPEG2TransportStreamFramer" code...

I did give you some specific suggestions. I did not give you any code,
because I think (maybe even know) that the design is flawed and should
not be used no matter how you tweak it, and I have therefor not
bothered to do any work on it, just rewrote the whole thing.

> In any case, I would still like to see a specific example of a
> (real-live) Transport Stream file that illustrates this problem.

If you have a satellite tuner card, and access to Hotbird 13E, you
will find tons of channels like I described. Don't think I am allowed
to distribute those streams though.
If you cannot see Hotbird, you could probably find tons of free to air
channels from www.lyngsat.com

> I don't buy this, because the streaming itself causes the data to be
> read linearly.

Yes, if you have only one client, but having 30 clients, none of them
at the same point in the stream, or maybe even on different streams,
causes random access on the disk. Reading 188 byte here, and then 188
byte there, often at totally different places on the disk is not very
efficient because of the long delay when moving the HDD heads around.
Random access is not efficient. If you read in larger bolks, you will
get a higher troughput from your HDD. Look at some of the tests at
storagereview, it is clear that reading smaller blocks spread around
the disk is very much slower than large blocks, and it is logical too,
if you think about it. If you clean up you house one place at a time
instead of just one small spot at a time, you spend less time running
around from place to place, and more time getting the job done.

> I think that a better (but related) solution would be to use the
> index file.  When we generate an index file, we are already 'reading
> ahead' in the stream, so it would be not be too difficult, I think,
> to augment the index file records to add an accurate 'duration' for
> each Transport Packet, and to use this 'duration' when streaming.
> Perhaps I'll add this at some point in the future...  (Of course,
> this won't help at all with live streams (because they don't have
> index files), but I think that having live Transport Streams that are
> severly VBR is a really bad idea, if you're not counting on clients
> to have sufficient buffering to handle them.)

This is what we are doing. As the solution is out in the open now, I
can tell you that it solves all the buffer problems. Combined with a
read-ahead routine we have, the efficiency of live555 is greatly
increased. Our code demands some changes to live that you have not
wanted to add earlier, as well as some custom changes that is not very
usefull for everyone. I do not believe that the changes we have will
make you very happy =/ You do have the solutions though, and it is
actually not very hard to do. You will need to index _every_ PCR
though. An average between I frames is not sufficient. I have tested
that.
By the way, the PCR is not supposed to be a guideline. The PCR
timestamp is the exact time the packet should reach the client. You
really have no slack there. It is an extremely accurate timestamp
because it needs to be. The accuracy of the PCR is how much slack you
should count on. If that is disregarded, then you are in for problems
sooner or later. Claiming that it is the client that should do it
differently does not help the guy that already have these (probably
even standard compliant) STBs that are strict with the PCR timings. I
do have real life experience from working in earth stations to back
that up. 50ms jitter could actually be too much, and even less on VBR.
If I have understood this correctly, the DVB standard was developed
with one very strong read thread going trough it all. STBs should be
cheap. As cheap as possible, putting all the cost on the distribution
system, giving a better total system cost, and lower end user prices.
That is still being done, and I think it is the right way of attacking
the problem with consumers not willing to pay a wole lot for stuff
like this. Being strict on PCR will lower the cost of parts and
manufacturing. It also, as I stated, improves interactivity response
times.

As for severly VBR, it is standard compliant, it is efficient, and
very many providers use this on entire muxes to give better average
quality/bit/s in their services. These dynamic VBR muxes can have huge
changes, very quickly, adapted on a per frame basis. I agree that CBR
is more equipment friendly, but is not what the real world provides
you with. Severly VBR is here to stay, and with good reason. Take any
DVD as an example. The bitrate fluctuations are totaly disregarded, it
is the quality that counts, and that is the way it should be.

The problem with live sources, I already said how you could fix that.
The live sources do of course not give data to fast or to slow, at it
is a live process being done realtime, it sends the data at the
exactly correct time (somtimes in big chunks that you have to even out
over time, but still, the timing is right). There is no reason to
re-time the packets in a framer, they are fine just the way they are.
Just drop the framer, and add support for setting the RTP timestamp to
server-time, and you're done. Whatever you have in store, it will
never beat the PCR accuracy from DVB-S, DVB-T and DVB-C sources
anyway. Just pass it on as soon as you receive it, introducing as
little jitter as possible. I have provided you with a patch before to
make this possible trough using timestamps instead of
durationInMicroseconds if one wished, but it was never added to
live555. I do as before suggest that it gets added. If you do not have
the patch anymore, I think I could dig it up again. It has been some
time since I last had a look at it.

Don't get me wrong here. Live555 is an excellent product, the best
there is imo, but it does have it's flaws. The transprtstremframer
works, it is just not very good at what it does, as opposed to most
other tings in live555. Having all my suggestions and experiences
disregarded because you think that the clients or providers should fix
the problems does not make me very eager to help out too much though.
Now that you have trick play implemented though, you will have alot of
new users, and transportstreamframer will need to be fixed for them to
be able to use it properly. At least we could not.


Best regards
-Morgan-