Hello, I need to setup Wowza to stream content from Amazon S3.
After digging through the documentation it seems to me that the only acknowledged way to do this is via an s3fs mount.
Can you please confirm that this is the only way Wowza can access files residing on a S3 bucket?
Do you provide (or plan to) some kind of integration API to achieve the same result?
I’m asking because the project I’m working on already has a lot of data on S3 organized in a “directory” hierarchy which s3fs does not recognise properly unless you do a bit of hacking.
Thank you in advance
Actually we don’t recommend using an s3fs mount for streaming content. Use the MediaCache Addon:
https://www.wowza.com/docs/how-to-scale-video-on-demand-streaming-with-media-cache
There is an example for Amazon S3 included in the /conf/MediaCache.xml file, including a place to put AWS keys so you can stream protected content.
Richard
Did you re-start Wowza after adding the keys? That is necessary.
Richard
Yes, if you are using the “vods3” application on EC2, you are using MediaCache.
Richard
Richard
I’ve been following several threads for some time now (on writing/reading to/from Amazon S3 buckets). I have a project coming up in the near future that could benefit from this, I think.
I’ll have a Wowza Server (Amazon EC2 - likely Wowza 2.X) running twice a week for a couple hours with 6-10 live streams (also being recorded).
Once these live streams are completed, I would like the streams to then get moved so a designated Amazon S3 bucket.
I have another Wowza server (Amazon EC2; Wowza 2.X) that runs 24/7. I would like this server to be able to handle any VOD3 requests from viewers (from the recordings moved to S3 from the temporary Wowza server). It’s extremely low traffic – maybe 10-20 simultaneous views at any given time.
My question is not in the setup – my question is in performance. I’ve seen Wowza Support (yourself included) mention several times on various threads that there are some “performance issues” to keep in mind when writing/reading to/from a S3 bucket.
Can you tell me what, exactly, these “performance issues” are? Is there a performance hit on the server (CPU hit or too many processes in queue, for example), or would it be a performance hit on the end-viewer’s side (such as a delay in watching the recorded video due to it having to be ‘loaded’ from S3)?
I understand that MediaCache is the recommended method of streaming the recordings, although I’m not sure why (again – because I don’t know what “performance issues” are, exactly).
I also see that the MediaCache AddOn is something you have to request – and I’m using Amazon EC2 pre-built images (Wowza 2.x), so I’m not sure how to go about requesting this as I don’t really have a “License Key” (do I?).
David L Good
Hi David,
… it would be best to update your Wowza instances to use the latest ami’s.
Recording directly to s3fs is also not recommended for the same reasons. You should record to a local folder and then use the file mover module to move the recording onto the s3fs partition once it is complete.
Roger.
Roger
I would like to move to a newer AMI, but for this particular group they really like CACTI… so that is the primary reason for sticking with Wowza 2.X AMIs.
So, just to clarify…
If I use the module that automatically writes a file to Amazon S3 (after the live stream is done recording), and then use VOD3 to serve the files from Amazon S3, MediaCache will automatically be used and I shouldn’t have any huge bottleneck issue. Would this be a correct assumption?
I’m not trying to record directly to S3 (only moving the recordings over once completed). And I’m fine with MediaCache handling the playback from S3.
David L Good
Hi David,
The Amazon AMI’s have MediaCache installed by default. The vods3 application that is preconfigured is setup to use MediaCache using the HTTP method to retrieve the content from s3. The MediaCache has been updated quite a bit since the 2.x Wowza ami’s so it would be best to update your Wowza instances to use the latest ami’s.
The Performance Issues when streaming from an s3fs mount is that is too slow. S3fs is a wrapper around http methods that are used to connect with s3 that makes it look like a normal partition. Vod streaming requires a fast read rate on the storage. When streaming vod, each player stream will be reading the file so reading directly from s3 using s3fs would result in a bottleneck very quickly. When you use MediaCache with http to connect to s3, only 1 copy of the file is retrieved and is cached locally for delivery.
Recording directly to s3fs is also not recommended for the same reasons. You should record to a local folder and then use the file mover module to move the recording onto the s3fs partition once it is complete.
Roger.
Thank you very much for the prompt response.
I’ve just downloaded the add-on and will get back with the results.
Regards,
Carlo
Hello, I was successful in installing the addon and get to stream public content from a bucket, however I can’t seem to be able to stream private content even after setting the key/secret pair in MediaCache.xml.
I keep getting:
WARN server comment - MediaCacheItemHTTPImpl.getItemInfo[dev-wowza.s3-eu-west-1.amazonaws.com/7af1817b09ec441a64318a58e038762eb33f285a.mp3]: HTTP response: 403
I’d like to point out that I triple-checked the key/secret value and the correct positioning in the XML tree.
Am I missing something here?
TIA
Carlo
Yes, I usually restart the server whenever I change something in the configuration files, just to be sure.