We have a 2 origins receiving different rtmp stream ( each has its own encoder pushing) and two edges in front of those origins. The whole set up is accelerated by a CDN.
We are running HDS and HLS with DVR.
I am trying to simulate encoder failure to origin 1. Normally the edges would search for the presence of a live fresh stream on the origin 2 and switch. However, since we have DVR, the edge never sees a failure as the origin1 happily returns the now (not live) dvr chunks… and no switch.
How to do i force the origins to try both edges for a live feed before failing back to the recorded stream??
Also, in a separate but related thread, we normally sync the rtmp streams (FMLE) by sending unix timestamps so that when switching the switch is smooth. In this scenario we are using FFMPEG, does a flag exist for sending similar metadata?
I eventually fixed the issue by having two applications on each origin. The first application receives the stream and push publishes to both itself and the origin. Same for both origins. I set up a don’t accept new connections if stream is present on the final application.
Push to both origins, but start primary encoder FIRST.
This means that the DVR applications are receiving identical streams which only very slightly differences in timing and DVR chunking should occur to the nearest key frame.
You now have identical chunks on each server.
Set up varnish(s) and use those as the edges.
Make sure to turn off Randomise Chunk Names and keep the cache-controls low so that CDN/varnish dont fill up with oild chunks after encoder restarts.
The system will suffer encoders loss AS LONG AS there is always one pushing.