DVR existing chunk returns 404 response

(sorry for my bad english)

Hello

We’ve started using wowza not so long ago and now we have one trouble

(i use trial wowza account and 3.6.2 version if it is important)

im’ streaming rtmp stream to application with trascoder (with transrate template) and nDVR enabled in it

when streming is on and i’m watchin’ from …/ngrp:stream_all/manifest.f4m?DVR all is ok, but

when streaming is done and this url is no longer avaible i create smil file with all streams. for some time all is ok, but then i periodically recieve 404 response on flash player chunck request (watching in fiddler).

there is this record in wowza log:

2013-06-28 08:09:01 UTC play stream INFO 200 test_8_ac1.smil - defaultVHost out definst 1.04 XXX 80 http://XXX:80/out/smil:test_8_ac1.smil/manifest.f4m?DVR 89.249.161.75 http (sanjose) http://XXX/Content/plugins/player.swf Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.52 Safari/537.36 OPR/15.0.1147.100 (Edition Next) 392042129 0 6501 3 0 0 0 test_8_ac1.smil DVR - - - - http://XXX:80/out/smil:test_8_ac1.smil/manifest.f4m?DVR http://XXX:80/out/smil:test_8_ac1.smil/manifest.f4m DVR http://XXX:80/out/smil:test_8_ac1.smil/manifest.f4m DVR

2013-06-28 08:17:11 UTC comment server INFO 200 - DvrStreamManagerBase [out/definst/test_8_ac1_90p] : Timeout occurred. Destroying DVR recorder. - - - 2682.086 - - - - - - - - - - - - - - - - - - - - - - - - -

2013-06-28 08:17:11 UTC comment server INFO 200 - DvrStreamManagerBase [out/definst/test_8_ac1_90p] : Timeout occurred. Destroying DVR packetizer. - - - 2682.088 - - - - - - - - - - - - - - - - - - - - - - - - -

2013-06-28 08:17:11 UTC comment server INFO 200 - DvrStreamManagerBase.removeStreamStore[out/definst/test_8_ac1_90p] : stream:{DvrStreamStoreBase streamName:test_8_ac1_90p.0 isLoaded:true canPlay:true canRecord:true hasEncryption:false isRecording:false isRecordingPaused:false props:{Properties: dvrSanJosePlaylistVersion: “1.0”, isDvrPacketizer: true}]. - - - 2682.09 - - - - - - - - - - - - - - - - - - - - - - - - -

2013-06-28 08:17:11 UTC comment server INFO 200 - MediaStreamMap.removeLiveStreamPacketizer[out/definst/test_8_ac1_90p]: Destroy live stream packetizer: dvrstreamingpacketizer - - - 2682.092 - - - - - - - - - - - - - - - - - - - - - - - - -

2013-06-28 08:17:12 UTC comment server WARN 200 - HTTPStreamerAdapterBase.getDvrStores: DVR Stream not found test_8_ac1_90p - - - 823.534 - - - - - - - - - - - - - - - - - - - - - - - - -

2013-06-28 08:17:12 UTC comment server ERROR 500 - HTTPStreamerAdapterSanJoseStreamer.onMediaFile: java.lang.IndexOutOfBoundsException: Index: 3, Size: 3 - - - 823.537 - - - - - - - - - - - - - - - - - - - - - - - - -

i’ve checked ndvr storage for this chunk’s fragment - it was there and when i replay original 404 chunk request in fiddler after 30 minutes it succeeds.

please give me any advice to resolve this problem, because my flash player resets playback after this 404 error(

Hi,

From the log you have posted it shows the nDVR store for the stream being removed

2013-06-28 08:17:11 UTC comment server INFO 200 - DvrStreamManagerBase.removeStreamStore[out/definst/test_8_ac1_90p] : stream:{DvrStreamStoreBase streamName:test_8_ac1_90p.0 isLoaded:true canPlay:true canRecord:true hasEncryption:false isRecording:false isRecordingPaused:false props:{Properties: dvrSanJosePlaylistVersion: “1.0”, isDvrPacketizer: true}]. - - - 2682.09 - - - - - - - - - - - - - - - - - - - - - - - - -

So when attempting to playback the content will not be there. It suggests that when the live stream drops the Store is removed

2013-06-28 08:17:11 UTC comment server INFO 200 - MediaStreamMap.removeLiveStreamPacketizer[out/definst/test_8_ac1_90p]: Destroy live stream packetizer: dvrstreamingpacketizer - - - 2682.092 - - - - - - - - - - - - - - - - - - - - - - - - -

It may be that you have the archive strategy set to delete ?

Andrew.

I tested this and did not have any problem,

  • I configured an application for nDVR and Transcoder.

  • I started a stream with FMLE on that application.

  • I tested in the Wowza DVR example FlashHTTPPlayer to playback the ngrp url while the stream was running

  • I created a smil file from the ngrp while the stream was live following this guide

  • I copied the generated smil to the /content folder.

  • I stopped the live encoder.

  • Using the same player, I changed the url to use the new smil, and it played back as expected

  • See if those steps work for you, otherwise tell me what is different about what you are doing.

    Rihcard

Hello, thx for your reply

From the log you have posted it shows the nDVR store for the stream being removed

So when attempting to playback the content will not be there. It suggests that when the live stream drops the Store is removed

i’ve finished live streaming about 12h ago, so there was no live streaming at moment of this problem occured

and if i wait for some time problem with THIS chunk goes away, but there always is another chunk, with same problem

It may be that you have the archive strategy set to delete ?

i am sure it’s append - append - is in config

i have an idea that it’s my network drive problem, is there any way to add andvanced logging to wowza?

Alex

i have an idea that it’s my network drive problem

it seems strange to me that i have no problems with ngrp:…?DVR (when live streaming is on and i’m watching dvr) but always have some with smil:…?DVR with same streams (only checked with live steaming off)

I tested this and did not have any problem,

  • I configured an application for nDVR and Transcoder.

  • I started a stream with FMLE on that application.

  • I tested in the Wowza DVR example FlashHTTPPlayer to playback the ngrp url while the stream was running

  • I created a smil file from the ngrp while the stream was live following this guide

  • I copied the generated smil to the /content folder.

  • I stopped the live encoder.

  • Using the same player, I changed the url to use the new smil, and it played back as expected

  • See if those steps work for you, otherwise tell me what is different about what you are doing.

    Rihcard

    I’ll make clean wowza setup on holidays and check this steps out and post results in the beginning of next week

    by now i have logs in DEBUG mode from machine that’s not working properly, here is the place i’m talking about:

    2013-07-04	02:19:05	UTC	comment	server	DEBUG	200	-	HTTPStreamerAdapterMPEGDashStreamer.canHandle[origin_transcode/smil:icap_test_6_ac0.smil/DVR_b550000_w291576756_qRFZS.abst/Seg1-Frag74]: false	-	-	-	1645.064	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2013-07-04	02:19:05	UTC	comment	server	DEBUG	200	-	HTTPStreamerAdapterCupertinoStreamer.canHandle[origin_transcode/smil:icap_test_6_ac0.smil/DVR_b550000_w291576756_qRFZS.abst/Seg1-Frag74]: false	-	-	-	1645.065	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2013-07-04	02:19:05	UTC	comment	server	INFO	200	-	HTTPStreamerAdapterSanJoseStreamer.canHandle[origin_transcode/smil:icap_test_6_ac0.smil/DVR_b550000_w291576756_qRFZS.abst/Seg1-Frag74]: true	-	-	-	1645.065	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2013-07-04	02:19:05	UTC	comment	server	INFO	200	-	HTTPStreamerAdapterSanJoseStreamer.service: origin_transcode/smil:icap_test_6_ac0.smil/DVR_b550000_w291576756_qRFZS.abst/Seg1-Frag74	-	-	-	1645.066	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2013-07-04	02:19:05	UTC	comment	server	INFO	200	-	HTTPStreamerAdapterSanJoseStreamer.onMediaFile: origin_transcode/smil:icap_test_6_ac0.smil/DVR_b550000_w291576756_qRFZS.abst/Seg1-Frag74	-	-	-	1645.067	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2013-07-04	02:19:05	UTC	comment	server	INFO	200	-	HTTPStreamerAdapterSanJoseStreamer.getDvrSanJoseFragments: get DVR for A/V 73.	-	-	-	498.964	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2013-07-04	02:19:07	UTC	comment	server	INFO	200	-	DvrStreamManagerBase [origin_transcode/_definst_/icap_test_6_ac0_450p] : Timeout occurred.  Destroying DVR recorder.	-	-	-	1646.724	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2013-07-04	02:19:07	UTC	comment	server	INFO	200	-	DvrStreamManagerBase [origin_transcode/_definst_/icap_test_6_ac0_450p] : Timeout occurred.  Destroying DVR packetizer.	-	-	-	1646.724	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2013-07-04	02:19:07	UTC	comment	server	INFO	200	-	DvrStreamManagerBase.removeStreamStore[origin_transcode/_definst_/icap_test_6_ac0_450p] : stream:{DvrStreamStoreBase streamName:icap_test_6_ac0_450p.0 isLoaded:true canPlay:true canRecord:true hasEncryption:false isRecording:false isRecordingPaused:false props:{Properties: isDvrPacketizer: true}].	-	-	-	1646.725	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2013-07-04	02:19:07	UTC	comment	server	INFO	200	-	MediaStreamMap.removeLiveStreamPacketizer[origin_transcode/_definst_/icap_test_6_ac0_450p]: Destroy live stream packetizer: dvrstreamingpacketizer	-	-	-	1646.725	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2013-07-04	02:19:08	UTC	comment	server	DEBUG	200	-	HTTPStreamerAdapterMPEGDashStreamer.canHandle[origin_transcode/smil:icap_test_6_ac0.smil/DVR_b550000_w291576756_qRFZS.abst/Seg1-Frag75]: false	-	-	-	1647.595	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2013-07-04	02:19:08	UTC	comment	server	DEBUG	200	-	HTTPStreamerAdapterCupertinoStreamer.canHandle[origin_transcode/smil:icap_test_6_ac0.smil/DVR_b550000_w291576756_qRFZS.abst/Seg1-Frag75]: false	-	-	-	1647.595	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2013-07-04	02:19:08	UTC	comment	server	INFO	200	-	HTTPStreamerAdapterSanJoseStreamer.canHandle[origin_transcode/smil:icap_test_6_ac0.smil/DVR_b550000_w291576756_qRFZS.abst/Seg1-Frag75]: true	-	-	-	1647.596	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2013-07-04	02:19:08	UTC	comment	server	INFO	200	-	HTTPStreamerAdapterSanJoseStreamer.service: origin_transcode/smil:icap_test_6_ac0.smil/DVR_b550000_w291576756_qRFZS.abst/Seg1-Frag75	-	-	-	1647.596	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2013-07-04	02:19:08	UTC	comment	server	INFO	200	-	HTTPStreamerAdapterSanJoseStreamer.onMediaFile: origin_transcode/smil:icap_test_6_ac0.smil/DVR_b550000_w291576756_qRFZS.abst/Seg1-Frag75	-	-	-	1647.596	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2013-07-04	02:19:08	UTC	comment	server	WARN	200	-	HTTPStreamerAdapterBase.getDvrStores: DVR Stream not found icap_test_6_ac0_450p	-	-	-	501.493	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2013-07-04	02:19:08	UTC	comment	server	ERROR	500	-	HTTPStreamerAdapterSanJoseStreamer.onMediaFile: java.lang.IndexOutOfBoundsException: Index: 2, Size: 2	-	-	-	501.494	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    
    
    • HDS client have 404 chunk response

    later application stops

    02:55:08	UTC	app-stop	application	INFO	200	_definst_	origin_transcode/_definst_	-	-	-	3807.858	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    

    then i try to play stream once more - this time successfully

    2013-07-04	03:07:42	UTC	comment	server	DEBUG	200	-	HTTPStreamerAdapterMPEGDashStreamer.canHandle[origin_transcode/smil:icap_test_6_ac0.smil/DVR_b550000_w1752937226_qRFZS.abst/Seg1-Frag74]: false	-	-	-	4562.165	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2013-07-04	03:07:42	UTC	comment	server	DEBUG	200	-	HTTPStreamerAdapterCupertinoStreamer.canHandle[origin_transcode/smil:icap_test_6_ac0.smil/DVR_b550000_w1752937226_qRFZS.abst/Seg1-Frag74]: false	-	-	-	4562.166	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2013-07-04	03:07:42	UTC	comment	server	INFO	200	-	HTTPStreamerAdapterSanJoseStreamer.canHandle[origin_transcode/smil:icap_test_6_ac0.smil/DVR_b550000_w1752937226_qRFZS.abst/Seg1-Frag74]: true	-	-	-	4562.167	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2013-07-04	03:07:42	UTC	comment	server	INFO	200	-	HTTPStreamerAdapterSanJoseStreamer.service: origin_transcode/smil:icap_test_6_ac0.smil/DVR_b550000_w1752937226_qRFZS.abst/Seg1-Frag74	-	-	-	4562.167	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2013-07-04	03:07:42	UTC	comment	server	INFO	200	-	HTTPStreamerAdapterSanJoseStreamer.onMediaFile: origin_transcode/smil:icap_test_6_ac0.smil/DVR_b550000_w1752937226_qRFZS.abst/Seg1-Frag74	-	-	-	4562.168	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2013-07-04	03:07:42	UTC	comment	server	INFO	200	-	HTTPStreamerAdapterSanJoseStreamer.getDvrSanJoseFragments: get DVR for A/V 73.	-	-	-	145.211	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2013-07-04	03:07:44	UTC	comment	server	DEBUG	200	-	HTTPStreamerAdapterMPEGDashStreamer.canHandle[origin_transcode/smil:icap_test_6_ac0.smil/DVR_b550000_w1752937226_qRFZS.abst/Seg1-Frag75]: false	-	-	-	4564.468	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2013-07-04	03:07:44	UTC	comment	server	DEBUG	200	-	HTTPStreamerAdapterCupertinoStreamer.canHandle[origin_transcode/smil:icap_test_6_ac0.smil/DVR_b550000_w1752937226_qRFZS.abst/Seg1-Frag75]: false	-	-	-	4564.468	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2013-07-04	03:07:44	UTC	comment	server	INFO	200	-	HTTPStreamerAdapterSanJoseStreamer.canHandle[origin_transcode/smil:icap_test_6_ac0.smil/DVR_b550000_w1752937226_qRFZS.abst/Seg1-Frag75]: true	-	-	-	4564.468	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2013-07-04	03:07:44	UTC	comment	server	INFO	200	-	HTTPStreamerAdapterSanJoseStreamer.service: origin_transcode/smil:icap_test_6_ac0.smil/DVR_b550000_w1752937226_qRFZS.abst/Seg1-Frag75	-	-	-	4564.469	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2013-07-04	03:07:44	UTC	comment	server	INFO	200	-	HTTPStreamerAdapterSanJoseStreamer.onMediaFile: origin_transcode/smil:icap_test_6_ac0.smil/DVR_b550000_w1752937226_qRFZS.abst/Seg1-Frag75	-	-	-	4564.469	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2013-07-04	03:07:44	UTC	comment	server	INFO	200	-	HTTPStreamerAdapterSanJoseStreamer.getDvrSanJoseFragments: get DVR for A/V 74.	-	-	-	147.512	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2013-07-04	03:07:47	UTC	comment	server	DEBUG	200	-	HTTPStreamerAdapterMPEGDashStreamer.canHandle[origin_transcode/smil:icap_test_6_ac0.smil/DVR_b550000_w1752937226_qRFZS.abst/Seg1-Frag76]: false	-	-	-	4566.866	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2013-07-04	03:07:47	UTC	comment	server	DEBUG	200	-	HTTPStreamerAdapterCupertinoStreamer.canHandle[origin_transcode/smil:icap_test_6_ac0.smil/DVR_b550000_w1752937226_qRFZS.abst/Seg1-Frag76]: false	-	-	-	4566.866	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2013-07-04	03:07:47	UTC	comment	server	INFO	200	-	HTTPStreamerAdapterSanJoseStreamer.canHandle[origin_transcode/smil:icap_test_6_ac0.smil/DVR_b550000_w1752937226_qRFZS.abst/Seg1-Frag76]: true	-	-	-	4566.867	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2013-07-04	03:07:47	UTC	comment	server	INFO	200	-	HTTPStreamerAdapterSanJoseStreamer.service: origin_transcode/smil:icap_test_6_ac0.smil/DVR_b550000_w1752937226_qRFZS.abst/Seg1-Frag76	-	-	-	4566.867	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2013-07-04	03:07:47	UTC	comment	server	INFO	200	-	HTTPStreamerAdapterSanJoseStreamer.onMediaFile: origin_transcode/smil:icap_test_6_ac0.smil/DVR_b550000_w1752937226_qRFZS.abst/Seg1-Frag76	-	-	-	4566.868	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2013-07-04	03:07:47	UTC	comment	server	INFO	200	-	HTTPStreamerAdapterSanJoseStreamer.getDvrSanJoseFragments: get DVR for A/V 75.	-	-	-	149.911	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    

    without changing anything

i’ve done that:

  • created new Windows Azure VM with Windows Server 2012 Datacenter installed

  • installed .NET Framework 3.5

  • installed jdk-7u25-windows-x64

  • installed WowzaMediaServer-3.6.2

  • configured nDVR + transcoder app

  • streamed for some time (2h actually) (with not FMLE but XSplit, but in previous tests I’v used FMLE, so it seems to me it doesn’t matter)

  • created smil from ngrp with guide you gave in prev post while it was live

  • start playing HDS smil stream with player based on OSMF from the beginning and on Seg1Frag118 got 404 (saw in fiddler) and following error in wowza console.

  • restarted wowza, replay request with fiddler - got chunk with 200 status, not 404 :confused: