How To: Many live streams, many instances, predictable stream URLs?

Wowza Streaming Engine
1

Hi all, bit of a new comer to Wowza infrastructure, looking to solve a potential use case.

I have many Wowza instances running, serving my application. From the application, I have many users creating and sending up live streams over RTMP. The incoming streams are routed through AWS ELB to the individual instances. Currently those streams are then archived once completed, and dropped into an S3 bucket.

I now have each instance configured to expose those incoming streams via HLS, and using the public DNS of an individual instance, I can watch those streams live via HLS or RTMP.

Now, the tricky part, I’m unsure how to expose those streams, spread out over many Wowza instances, via a predictable url, rather than the individual instance public DNS.

I can imagine building a load balancer service that is aware of which specific instance a stream is routed to and can respond to the client with a specific URL, but was hoping something was built into Wowza ecosystem for this specific many instance/many stream purpose.

2

I did a bit more digging, and I think I have an idea of how to make this work, but not fully. The complexity seems to be from my streams being short lived and dynamically named. The system has thousands of dynamically named streams per day, and while they all need to be archived, as we’re doing today, only some will ever have someone actually attempt to watch them.

If I understand correctly, I could create a new ELB and set of Wowza instances that are specifically for watching.

Those instances could then be configured as live repeaters, using the ModuleStreamResolver module. The new set of “Watching” servers can then be configured in that module to have a list of all Origin servers, e.g. the existing set of instances I’m using today to receive/encode and archive incoming streams. The StreamResolver module, when receiving requests to watch a stream, would go check all the specified Origin servers to find the stream for playback, and forward it on.

However, there are some big gaps in my understanding. I’m not sure how the StreamResolver interacts with the Live Repeater concept, since Live Repeaters seem to be specifically configured for single origin, not many origin, environments. Do I need to do anything to the edge servers, other than just configure the StreamResolver module?

On top of that, as my encoding/origin servers are hosted in an ASG inside AWS, they’re completely dynamic and spun up/down as needed to support load. From what I can tell, StreamResolver only accepts a hard coded list of servers, so I don’t know how to update the playback servers to see (or remove) origin servers as they’re created/destroyed.

Another option might be for me to create my own microservice to assign incoming streams to a given encoder/origin instance, as well as record and respond to requests from other clients looking to view those streams.

Am I overthinking this/missing something?

3

Sorry you didn’t get a response @Tj_Fallon , Wowza had limited staff over the holiday break. Not sure if you submitted a ticket already, but the engineers would be happy to share some ideas with you based on other customers’ infrastructure. It can indeed get overwhelming with so many stream names being generated. I don’t have any tutorials to share on that stage of the process you are asking about though.

Wowza has two engineers here who helped write the StreamResolver module for Engine and for the GitHub sample code, so they’ll be able to give you direction. It’s considered a more advanced workflow so I don’t think this is you overthinking it, it’s not your simple stream configuration. Tech support would be happy to help once they get a better understanding of your needs.

They’ll want to review your complete config and logs though, but you can easily submit that through a support ticket here:

https://www.wowza.com/support/open-ticket