# Record stream directly to S3 with AWS Multipart Upload? **URL:** \ **Category:** Wowza Streaming Engine\ **Created:** [December 1, 2010, 10:14pm UTC](https://community.wowza.com/t/record-stream-directly-to-s3-with-aws-multipart-upload/34370 "2010-12-01T22:14:44Z")\ **Posts on this page:** 11\ **Page:** 2
**Author:** ![Colin\_Angevine](https://avatars.discourse-cdn.com/v4/letter/c/b5e925/32.png) [@Colin\_Angevine](https://community.wowza.com/u/Colin_Angevine)\ **Post date:** [December 2, 2010, 8:35am UTC](https://community.wowza.com/t/record-stream-directly-to-s3-with-aws-multipart-upload/34370/21 "2010-12-02T08:35:22Z")
Thanks for getting back to me so quickly, Charlie. You’re good! I know that S3FS is used to transfer the files from EC2 to S3, but I thought that the current implementation that is bundled with Wowza first records the FLV to the EC2 instance and detects when the recording is complete. When the recording has finished, it transfers the complete FLV to S3. Is that right? Is there any way that this process could be simplified by recording directly to the S3 bucket? And programs wouldn’t then need to record the file and then wait while the file is being transferred to S3 before playback. ---
**Author:** ![Colin\_Angevine](https://avatars.discourse-cdn.com/v4/letter/c/b5e925/32.png) [@Colin\_Angevine](https://community.wowza.com/u/Colin_Angevine)\ **Post date:** [December 2, 2010, 9:24am UTC](https://community.wowza.com/t/record-stream-directly-to-s3-with-aws-multipart-upload/34370/22 "2010-12-02T09:24:10Z")
Thanks, Charlie. ---
**Author:** ![dfs\_dfsasa](https://avatars.discourse-cdn.com/v4/letter/d/b77776/32.png) [@dfs\_dfsasa](https://community.wowza.com/u/dfs_dfsasa)\ **Post date:** [April 4, 2011, 9:47pm UTC](https://community.wowza.com/t/record-stream-directly-to-s3-with-aws-multipart-upload/34370/23 "2011-04-04T21:47:09Z")
Should I be able to see s3 in FileZilla? ---
**Author:** ![dfs\_dfsasa](https://avatars.discourse-cdn.com/v4/letter/d/b77776/32.png) [@dfs\_dfsasa](https://community.wowza.com/u/dfs_dfsasa)\ **Post date:** [April 5, 2011, 3:56pm UTC](https://community.wowza.com/t/record-stream-directly-to-s3-with-aws-multipart-upload/34370/24 "2011-04-05T15:56:40Z")
It’s mounted with s3fs. Everything is working fine except I cant see the ‘s3’ directory in FileZilla. What do you mean within FTP path? ---
**Author:** ![David\_Good](https://avatars.discourse-cdn.com/v4/letter/d/d9b06d/32.png) [@David\_Good](https://community.wowza.com/u/David_Good)\ **Post date:** [January 25, 2011, 5:52pm UTC](https://community.wowza.com/t/record-stream-directly-to-s3-with-aws-multipart-upload/34370/25 "2011-01-25T17:52:37Z")
Richard I’m a rookie, just FYI. I have followed the instructions above and added an “init.sh” file to the root of my .ZIP archive (startup package – same version you used). I added the following: #!/bin/sh mkdir /mnt/s3 /usr/bin/s3fs [cogwareplayhouston.s3.amazonaws.com](http://cogwareplayhouston.s3.amazonaws.com) -o accessKeyId=XXXXX -o secretAccessKey=XXXXX -o default\_acl=public-read /mnt/s3 Obviously the “XXXXX” would be my keys. The problem I’m having is that when I look at the launched instance I don’t see that a “s3” directory has been created in the “mnt” directory. Isn’t this script supposed to create that directory? My goal is to simply have a recording automatically transfer from one place and copy over to my s3 bucket once it has completed. Any suggestions? ---
**Author:** ![David\_Good](https://avatars.discourse-cdn.com/v4/letter/d/d9b06d/32.png) [@David\_Good](https://community.wowza.com/u/David_Good)\ **Post date:** [January 26, 2011, 9:02am UTC](https://community.wowza.com/t/record-stream-directly-to-s3-with-aws-multipart-upload/34370/26 "2011-01-26T09:02:45Z")
I’m using the ami-5453a53d Amazon EC2 Amazon Machine Image ID. In the startup package (based off the ‘startup\_2.2’ provided example) I modify the “startup.xml” file: * * * wowza * * * I then create an “init.sh” file and place it at the same level as the “startup.xml” file and “wowza” and “tuning” directories. The “init.sh” file looks like this (with the XXXXXXXXXX being replaced with my Amazon AWS keys: * * * #!/bin/sh mkdir /mnt/s3 /usr/bin/s3fs [cogwareplayhouston.s3.amazonaws.com](http://cogwareplayhouston.s3.amazonaws.com) -o accessKeyId=XXXXXXXXXX -o secretAccessKey=XXXXXXXXXX -o default\_acl=public-read /mnt/s3 * * * I have an S3 bucket called “cogwareplayhouston” that sits at the top level of my S3 account. I’m assuming that “[cogwareplayhouston.s3.amazonaws.com](http://cogwareplayhouston.s3.amazonaws.com)” is the correct way to point the script to this bucket, right? * * * I then used the instructions, as noted above by acropolis, found here: [https://www.wowza.com/docs/how-to-use-the-fuse-based-file-system-backed-by-amazon-s3](https://www.wowza.com/docs/how-to-use-the-fuse-based-file-system-backed-by-amazon-s3) I didn’t do the mkdir command or mount the S3 bucket from these instructions, since I assume those are handled by the “init.sh” What I did was modify the Wowza application that records video. In my case, I’m using “rtplive” to stream live video. I then added the following, exactly, to the end of the “Modules” list and the “Properties” list in the "Application.xml that sits in "conf/rtplive/. Since there were three “” areas I assumed it was the was the one right below the Modules list I just added to (just before “”). So, the final portion of the “Application.xml” looks like this: * * * ModuleMediaWriterFileMover ModuleMediaWriterFileMover com.wowza.wms.module.ModuleMediaWriterFileMover fileMoverDestinationPath /mnt/s3 fileMoverDeleteOriginal false Boolean fileMoverVersionFile true Boolean * * * I use the LiveStreamRecord example Flash player to “Start Recording” and then “Pause Recording”. When I FTP into the server I can see the “myStream.sdp.flv” file that is recorded, as well as the temporary file being written if I am still ‘recording’. When I’m done recording the temporary file goes away as it becomes part of the “myStream.sdp.flv” file that resides in the “content” folder. So, the file is correctly being recorded and saved to the “content” folder… but it’s not being moved to the s3 folder – in fact, when I FTP into the server I don’t see a “s3” folder anywhere, not even in the “mnt” directory. In the “mnt” directory I see “lost+found”, “mediacache” and “WowzaMediaServer” as the only directories. I’m not sure what I’m missing here. Any help would be greatly appreciated. ---
**Author:** ![David\_Good](https://avatars.discourse-cdn.com/v4/letter/d/d9b06d/32.png) [@David\_Good](https://community.wowza.com/u/David_Good)\ **Post date:** [January 26, 2011, 9:56am UTC](https://community.wowza.com/t/record-stream-directly-to-s3-with-aws-multipart-upload/34370/27 "2011-01-26T09:56:48Z")
Richard That was the issue. I changed the s3 location to “cogwareplayhouston” and everything worked like magic. Thanks for your help on this!!! ---
**Author:** ![Warren\_Gwartney](https://avatars.discourse-cdn.com/v4/letter/w/c57346/32.png) [@Warren\_Gwartney](https://community.wowza.com/u/Warren_Gwartney)\ **Post date:** [January 4, 2011, 1:39pm UTC](https://community.wowza.com/t/record-stream-directly-to-s3-with-aws-multipart-upload/34370/28 "2011-01-04T13:39:50Z")
I was trying to do something like this by creating an init.sh script and referencing it in the startup package. init.sh contains the following: #!/bin/sh mkdir /mnt/s3 /usr/bin/s3fs bucketname -o accessKeyId=accesskey -o secretAccessKey=secretkey -o default\_acl=public-read /mnt/s3 I wanted to share and maybe get some feedback. I haven’t been able to get it to work. When I try to launch and instance it’s completely broken. Might be a problem with my zip or encoding. Using a PHP script… er. trying to. ---
**Author:** ![Warren\_Gwartney](https://avatars.discourse-cdn.com/v4/letter/w/c57346/32.png) [@Warren\_Gwartney](https://community.wowza.com/u/Warren_Gwartney)\ **Post date:** [January 5, 2011, 8:21am UTC](https://community.wowza.com/t/record-stream-directly-to-s3-with-aws-multipart-upload/34370/29 "2011-01-05T08:21:45Z")
Thanks for the reply and for testing it out. Nice to put that part to rest. Now, referencing this page: [https://www.wowza.com/docs/how-to-use-the-fuse-based-file-system-backed-by-amazon-s3](https://www.wowza.com/docs/how-to-use-the-fuse-based-file-system-backed-by-amazon-s3) Great info. However, while it’s clear that one should not write directly to a mounted bucket, does the configuration described in the above article automatically move the file to the bucket? Assuming, yes; does this happen as soon as the stream is stopped? How does one know when it’s safe to unload and instance… meaning is there a way to get a status on the file move? It would be great if there was an http page that had a status on filemover activity. I am trying to automate an on demand webcast process and having to get on a command line doesn’t work well for that. Any pointers would be great. ---
**Author:** ![Savvas\_Panagiotidis](https://avatars.discourse-cdn.com/v4/letter/s/da6949/32.png) [@Savvas\_Panagiotidis](https://community.wowza.com/u/Savvas_Panagiotidis)\ **Post date:** [May 11, 2011, 3:05pm UTC](https://community.wowza.com/t/record-stream-directly-to-s3-with-aws-multipart-upload/34370/30 "2011-05-11T15:05:56Z")
Richard, unfortunately it doesn’t work on our case. We use EC2 with live-record and the module: ModuleMediaWriterFileMover ModuleMediaWriterFileMover com.wowza.wms.module.ModuleMediaWriterFileMover and the properties: fileMoverDestinationPath /mnt/s3 fileMoverDeleteOriginal false Boolean fileMoverVersionFile true Boolean It saves perfectly well the files on /mnt/s3, Now, if we “/usr/bin/s3fs mybucket -o accessKeyId=XXXXX -o secretAccessKey=XXXXXX -o default\_acl=public-read /mnt/s3”, and try to navigate to the /mnt/s3 folder it doesn’t allow us to even open the folder. Because of this, we are not able to use FME to stream. We have to reboot the machine for everything to work. Is there a chance for this be a permission issue or something else? Thank you Richard! ---
**Author:** ![Savvas\_Panagiotidis](https://avatars.discourse-cdn.com/v4/letter/s/da6949/32.png) [@Savvas\_Panagiotidis](https://community.wowza.com/u/Savvas_Panagiotidis)\ **Post date:** [May 11, 2011, 3:24pm UTC](https://community.wowza.com/t/record-stream-directly-to-s3-with-aws-multipart-upload/34370/31 "2011-05-11T15:24:57Z")
I spent all day trying to find what was the problem…! No EU buckets! [Previous page](https://community.wowza.com/t/record-stream-directly-to-s3-with-aws-multipart-upload/34370.md?page=1)