AdaptiveBitRate, nDVR, SMIL issue

I have the following issue.

Using Wowza 3.6.3 with Monthly license and ndvr addon. It has one stream with 3 bitrates sent with FMLE, the ndvr is configured using the above steps, with a window of 1 hour.

The addon works, I’ve tested all streams one by one with the provided test player and it works with manifest.f4m, also with playlist.m3u8 in hls compatible devices.

Then I created smil file for adaptive streaming, this also works on the hls devices, but here the problem appears.

For the past two weeks I started to use the feature continuously and after aprox. 2 days of streaming, the link with the smil stops working.

If I try to open the link:

http://streamserver.exampledomain.co…ylist.m3u8?DVR

it returns switch chunklist file like bellow

Code:

#EXTM3U

#EXT-X-VERSION:3

#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=1500000

http://streamserver.exampledomain.com:1935/DvrApplication1/definst/smil:dvrApplication1.smil/chunklist_w321055244_b1500000_DVR.m3u8

#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=500000

http://streamserver.exampledomain.com:1935/DvrApplication1/definst/smil:dvrApplication1.smil/chunklist_w321055244_b500000_DVR.m3u8

#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=3000000

http://streamserver.exampledomain.com:1935/DvrApplication1/definst/smil:dvrApplication1.smil/chunklist_w321055244_b3000000_DVR.m3u8

If click any of the links to the chunklist it tries to load it but it does not return the chunklist(the session is not expired I’m doing it in the same moment when it is returned).

Inside the error log this line is logged

Code:

HTTPStreamerAdapterCupertinoStreamer.service: Request timeout: 8000

If I open a link without smil, it returns the chunklist for everyone of the 3 bitrates.

In the interval before 2 days of continuous working, the smil link works as it should, returning chunklist.

Regards,

This happened again today, after aprox. 2 days of using the nDVR.

The direct links to the streams work

http://streamserver.exampledomain.com/ndvr/stream1/playlist.m3u8?DVR

http://streamserver.exampledomain.com/ndvr/stream2/playlist.m3u8?DVR

http://streamserver.exampledomain.com/ndvr/stream3/playlist.m3u8?DVR

Also they work with the delegate

http://streamserver.exampledomain.com/ndvr/stream1/playlist.m3u8?DVR&wowzadvrplayliststart=[timecode]

But the smil link does not work

http://streamserver.exampledomain.com/ndvr/smil:ndvr.smil/playlist.m3u8?DVR

http://streamserver.exampledomain.com/ndvr/smil:ndvr.smil/playlist.m3u8?DVR&wowzadvrplayliststart=[timecode]

I’ve edited the smil and removed one of the streams, and it returned the chunklist.

Why is this behavior happening, why the smil with 3 streams stops working after a while?

Zip up and send the /conf and /log folders to support@wowza.com. Tell us about when this last happened. It is possible that there was a network problem between the encoder and Wowza that interrupted one or more streams.

Is using the Wowza Transcoder an option? This will help because you will only need to send one high quality stream to Wowza, which the Transcoder will transrate to as many renditions as you configure and they will all be key frame aligned as necessary.

Also, for very long live events you might consider a mix of nDVR and .mp4 recordings made with the the Live Stream Record feature, which are more portable and better for archiving.

Richard

Latest test scenario was this.

It was streaming and recording with ndvr again for 2 days when my previous post was sent and I explain in that one how did I get the smil to work, with only two bitrates. And left the server to continue working without restarting it. Yesterday which is aprox. 4 days without restarting and continuous working, I checked the 3 bitrates smil and it did returned the chunklist and during the whole day it worked as it should. Seems to me some http resource on the streaming server is occupied which should return the 3 bitrate smil and after day or two becomes free again, resulting the 3 bitrate link to work again with nothing changed, it will be good to locate this with your assistance.

Inside the error log when 3 bitrate link is requested this is being logged, nothing else:

HTTPStreamerAdapterCupertinoStreamer.service: Request timeout: 8000

I’ll send you the conf shortly.

Server uses 2 core CPU with 3GB memory it is a 32bit machine, which for the current setup had no other issues except the above one.

There was no network interruption logged on server side nor the FMLE side, both server and encoder are on a production high bandwidth network which is constantly monitored. I would exclude this a factor because all else was working, returning a playlist/chunklist for live streams with smils, returning a playlist/chunklist for all dvr streams and returning a playlist/chunklist for live streams with 2 bitrate smil.

Your advice for combined recording is already implemented, we are recording 6 hours of tv programs in mp4 files and the last hour is recorded with ndvr to allow pausing the currently played streams.

Meanwhile I’ll reboot and leave it to work and start up another server in parallel with same setup to see what happens.

Regards,

Wowza running since the last post and the nDVR also. In the last setup I’m not using the smil with 3 bitrates, instead I’m using a smil with single bitrate.

***I know that in this case the smil can be left out because it is using only one bitrate but for testing purpouse to remove the http handeler for smil files as issue.

This means that the issue I have is one level deeper and I believe it is in the generating the dvr playlists for the smil request in conjunction with the server resources. From my opinion 3 bitrate smil will require from the system to generate 3 different playlists for each of the streams, 1 bitrate requires a single playlist and if I have about 40 test users requesting the 3 bitrate playlist it causing I beleive the dvr packetizer not been able to generate them all maybe due to a insufficient server resources on the other hand 1 bitrate smil can respond with a 1 playlist.

Not to push this in one direction that the issue is the number of connections vs system resoures, maybe we are overlooking something. Results from the second identical machine is that it works all the time with 3 bitrate smil, this machine we are using for development with around 3-4 users.

Does some of this is close to a logic conclusion? Any other ideas will be helpful so I can try different test case.

any news in this theme ?