Wowza severs and AWS cloud front caching

Wowza Streaming Engine
1

Hi there,

I’m now using wowza origin-edge architecture to do live streaming. Currently, my system looks like: two encoders->two Wowza origin servers->two edge servers->AWS ELB->AWS cloud front. Our system needs to be very robust so I put redundancy in every layer.

However, I found that every time a master playlist request is sent to the edge server, the edge server returns a master playlist as follow,

#EXTM3U

#EXT-X-VERSION:3

#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=250000

chunklist_w4135965.m3u8

if I further request the chunklist_w4135965.m3u8 media playlist, it returns

#EXTM3U

#EXT-X-VERSION:3

#EXT-X-ALLOW-CACHE:NO

#EXT-X-TARGETDURATION:10

#EXT-X-MEDIA-SEQUENCE:2140

#EXTINF:9.009,

media_w4135965_2140.ts

#EXTINF:9.009,

media_w4135965_2141.ts

the sequence w4135965 (do you call it session id?) seems to be random and different to any client. This makes the segments not cacheable.

Even if I replace the origin-edge servers with Wowza Http origin servers according to this article: https://www.wowza.com/docs/how-to-configure-a-wowza-server-as-an-http-caching-origin, the Wowza servers still return playlists containing that sequence. And for a Wowza server, the sequence is fixed but for different servers, their sequences are still different. As you see, in our system, we need multiple origin servers and the random sequences really suck.

Hope someone could solve my problem. :slight_smile:

Thanks!

Yorick

2

Hello,

Thank you for participating in the Wowza Forums. You could create a custom module that sets a server id cookie. With the standard ELB sticky sessions where ELB sets the cookie, it also adds a header (no-cache=“set-cookie”) that tells CloudFront to not cache the cookie so when the request goes back to the ELB, the cookie is missing and ELB just sends the request to the next server. By setting and using a server id cookie, the ELB doesn’t set the no-cache header when the cookie is set and everything appears to work properly. What you want is for individual stream requests to be balanced across the pool but then have the chunk requests to hit the proper servers. The only way to do this is to use server id cookies.

The code is pretty simple. Create the onHTTPSessionCreate method as a point to set a user header.

public void onHTTPSessionCreate(IHTTPStreamerSession httpSession) { 
String cookie = "serverId="+Server.getInstance().getGUID()+";PATH=/;MAX-AGE=60"; 
httpSession.setUserHeader("Set-Cookie", cookie); 
}

This will set the Set-Cookie header on every response for the http stream. It may not be the cleanest approach, but it is not really easy to intercept each individual request to check this. From a cache and player perspective, it doesn’t really harm anything.

In the ELB, select the option to use an application cookie, setting it to look at the serverId cookie. With the ELB set this way, it doesn’t set the additional Cache-Control header that prevents the cookies from being cached.

In CloudFront, set it to use all cookies. The only ones that should be involved are the serverId cookie and ELB’s own session cookie.

You could also increase the max-age setting on the *CacheControlPlaylist properties in the Application.xml if this is vod http origin mode, the playlists do not change unless the source changes so the playlist can be cached. Because the initial playlist request will not have the cookie set, the cached response will set the cookie which will then be used for the rest of the stream.

As always, test this scenario thoroughly prior to release into your production environment.

Regards,

Mac

3

Hi,

If you have configured http origin mode correctly then the session id sequence will be removed from the urls. That is the purpose of http origin mode. Make sure you have restarted the wowza application on each of the servers after configuring http origin mode.

The process that Mac has described above is to get CloudFront and ELB and Wowza working together properly. Without the custom cookie, the ELB will forward every segment request to a different server. This creates extra work for the servers and can also cause synchronization problems if the timing of the streams on each server are slightly different.

Roger.

4

Hi Mac,

Thank you so much for this detailed reply. It looks pretty reasonable. I’ll try it tomorrow! But I still have a question: in what Wowza module or with what API can I modify the playlist? If I can remove the random sequence from the segment URLs, I won’t suffer the trouble anymore. :smiley:

Thanks!

Yorick

5

Hi everyone,

Finally I figured it out. I checked the Application.xml of this application and found that even I configured it as “Http origin” using Wowza Engine Manager, the property “randomizeMediaName” is still true (so weird). After I manually changed it to false the output media list contains no random sequence any more. :wink:

/Wowza install directory/conf/application folder/Application.xml:

…

httpRandomizeMediaName

false

Boolean

…

Here is the output after this modification:

yshen-retina:live yshen$ curl http://Wowza server ip:1935/application/smil file name/playlist.m3u8

#EXTM3U

#EXT-X-VERSION:3

#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=2470000

chunklist_b2470000.m3u8

#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=850000

chunklist_b850000.m3u8

#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=200000

chunklist_b200000.m3u8

yshen-retina:live yshen$ curl http://wowza server ip:1935/application/smil file name/chunklist_b200000.m3u8

#EXTM3U

#EXT-X-VERSION:3

#EXT-X-ALLOW-CACHE:NO

#EXT-X-TARGETDURATION:15

#EXT-X-MEDIA-SEQUENCE:109

#EXTINF:9.009,

media_b200000_109.ts

#EXTINF:12.012,

media_b200000_110.ts

#EXTINF:9.009,

media_b200000_111.ts

Another thing is that if the application is configured as “HttpOrigin”, the SMIL icon won’t exist in the Wowza Engine Manager. To do adaptive bitrate streaming, we have to create a SMIL file under the content folder by ourselves.

However, for some reason, our system cannot use Cookies. So although the solution proposed by Mac is really attractive, I cannot use it. For now, I plan to use Nginx server as a replacement of AWS ELB to help distribute client request to specific Wowza servers (using Nginx consistent hashing). To achieve this, I want the segment URLs to be like:

server_id/media_13.ts

server_id/media_14.ts

…

or media_13.ts?server_id=xxx

media_14.ts?server_id=xxx

…

where server_id is a identifier associated with a specific Wowza server.

In short, I still need to modify the media playlist…I’m reading the Wowza API references but it’ll be greatly appreciated if anyone could directly tell me how to modify it. :wink: