Frequest Media Cache time out errors

Wowza Streaming Engine
1

Hello all,

I am running a Wowza server which publishes 24 hour pseudo-live streams, using the Media Cache to pull in files from cloud storage as needed.

While monitoring the streams & logs, I see several skipped videos due to Media Cache timeouts. The most common is “Read timed out”, but I see an occasional “Connection reset” as well.

MediaCacheHTTPByteReader.sendRequest[http://www.someurl.com]: java.net.SocketTimeoutException: Read timed out
MediaCacheHTTPByteReader.sendRequest[http://www.someurl.com]: java.net.SocketException: Connection reset

This happens frequently enough to cause drift from my original playlists over time due to videos being skipped.

Are there changes I can make to the Media Cache config to improve this? What are the potential trade offs?

Here is what I see as the relevant section of MediaCache.xml (NOTE: This is for a Wowza 3.6 server, plan to upgrade to 4 next month)

<!-- maximum number of concurrent connections this source will make back to the web server -->
				<Property>
					<Name>maxPoolSize</Name>
					<Value>25</Value>
					<Type>Integer</Type>
				</Property>
			 	<!-- HTTP request timeout -->
				<Property>
					<Name>httpReadTimeout</Name>
					<Value>6000</Value>
					<Type>Integer</Type>
				</Property>
			  	<!-- HTTP connection timeout -->
				<Property>
					<Name>httpConnectionTimeout</Name>
					<Value>6000</Value>
					<Type>Integer</Type>
				</Property>
			  	<!-- HTTP read request retry count -->
				<Property>
					<Name>httpReadRetries</Name>
					<Value>3</Value>
					<Type>Integer</Type>
				</Property>
				<!-- Close socket connection if Content-Length incorrect on byte range request -->
				<Property>
					<Name>closeConnectOnBadContentLength</Name>
					<Value>false</Value>
					<Type>Boolean</Type>
				</Property>
			  	<!-- HTTP send buffer size -->
				<Property>
					<Name>httpSendBufferSize</Name>
					<Value>8000</Value>
					<Type>Integer</Type>
				</Property>
			  	<!-- HTTP receive buffer size -->
				<Property>
					<Name>httpReceiveBufferSize</Name>
					<Value>65000</Value>
					<Type>Integer</Type>
				</Property>

Thanks in advance

2

Hi Chris

Can you elaborate on your Media Cache configuration? Is this local or remote from your Wowza server? As you’ve shown it looks like there is some connectivity issue between one and the other. Can you carry out some simple tests to see if there are any network issues while running this configuration. A simple ping from the Wowza server to the HTTP store (if this is remote from the Wowza server) may indicate issues with connectivity for example.

Paul

3

Hi Chris,

The timeouts and resets you are seeing is due to the web server not responding to the requests quickly enough. It could possibly be due to the web server getting overloaded. You could try increasing the httpReadTimeout & httpConnectionTimeout values but if the remote server is overloaded, then it would just cause a longer delay before resetting.

MediaCache always makes byte range requests but it may still cause playback errors if the data cannot be retrieved.

Roger.

4

Hi Paul,

The MediaCacheStore is a local directory in the ServerConfigHome

The MediaCacheSource is remote - an HTTP location, using MediaCacheItemHTTPImpl

There is not a consistent network issue, as the Media Cache works effectively most of the time. It occurs maybe 10 times in 24 hours, and I haven’t seen consecutive or repeated failures.

I’m wondering if changing the readTimeout and/or retries settings would help?

Are these errors fired when the entire file cannot be retrieved, or when a single byte-range request fails? If the later, it may not be as big a problem as I thought