MediaCache error / Local Apache forward proxy with SSL

Wowza Streaming Engine
1

Hi,

I need to secure traffic between MediaCache and my origin web server. I’m needing assistance with the following solution to securing this traffic:

  • Edge server: I have both Wowza with MediaCache and apache 2.2 running on the same server called “edgeserverhost”

o Apache is configured as a forward proxy, accepting only local connections and forwarding them to the Origin server over SSL

  • “Origin” server: I have IIS 7.5 and Wowza running on the same server for “originserverhost”

o IIS is configured to accept SSL connections over port 1443 with an internally signed certificate (enterprise). No other special configuration was done except for adding MIME types to handle smil and mp4

This configuration does work properly, MediaCache traffic is secured over SSL. However, I’m noticing an odd issue when playing content off of my edge server.

Error log:

#Version: 1.0

#Start-Date: 2013-11-11 09:29:33 CST

#Software: Wowza Media Server 3.6.2.10 build6427

#Date: 2013-11-11

ERROR server comment 2013-11-11 09:29:33 ----- 0.308 -------- MediaCacheHTTPByteReader.sendRequest[http://edgeserverhost:8000/path/file.smil]: java.net.SocketException: Software caused connection abort: socket write error

ERROR server comment 2013-11-11 09:29:51 ----- 18.255 -------- MediaCacheHTTPByteReader.sendRequest[http://edgeserverhost:8000/path/file.mp4]: java.net.SocketException: Software caused connection abort: recv failed

Some notes:

  • This error also occurs when using v3.6.3

  • Servers are both running Windows 2008 R2, virtualized with VMWARE with 4vCPU/4GB/1Gig

  • Both servers exist on the same VMWare HostGroup, traffic never leaves the virtual switch.

  • Running jdk1.7.0_25

  • Running in a test environment, there is no production load on the servers, this happens when trying to load one item

Configuration:

MediaCache.xml

http1

http://edgehostname:8000/

http1/

com.wowza.wms.plugin.mediacache.impl.MediaCacheItemHTTPImpl

256K

86400000

7200000

true

25

false

Httpd.conf

ThreadsPerChild 250

MaxRequestsPerChild 0

LoadModule authz_host_module modules/mod_authz_host.so

LoadModule proxy_module modules/mod_proxy.so

LoadModule proxy_connect_module modules/mod_proxy_connect.so

LoadModule proxy_http_module modules/mod_proxy_http.so

LoadModule ssl_module modules/mod_ssl.so

Listen 8000

SSLSessionCache “shmcb:F:/APP/apache2/logs/ssl_scache(512000)”

SSLSessionCacheTimeout 300

<VirtualHost *:8000>

ProxyRequests Off

SSLProxyEngine On

ProxyPass / https://originhostname:1443/

Order deny,allow

Deny from all

Allow from edgeipaddress

2

Refer to ticket in progress: 72306

3

I’ve found that either the apache version, or the way Windows handles TCP connections destined for localhost were the root cause. I wasn’t able to see the connection process on the windows server because none of the tools I have available are able to capture localhost traffic.

Windows server: Wowza 3.6.3 with Apache/2.2.6 (Win32)

Errors reported while streaming “Software caused connection abort: recv failed”

Performance was poor.

Linux server: Wowza 3.6.3 with Apache/2.2.22 (Ubuntu)

Successful, no errors.

Performance wasn’t negatively impacted.

*Using the same apache modules and configuration on both servers.

*For the Windows server, the errors were only alleviated when I stood up an Apache reverse proxy on a completely separate windows server, and pointed Wowza to it. This is what leads me to deduce that the two operating systems treat localhost TCP connections completely differently.

::Working apache config on Linux::

::You’ll want only the following modules::

a2enmod authz_host cache mem_cache proxy proxy_connect proxy_http ssl

LockFile ${APACHE_LOCK_DIR}/accept.lock

PidFile ${APACHE_PID_FILE}

Timeout 300

ServerName hulk

KeepAlive On

MaxKeepAliveRequests 100

KeepAliveTimeout 5

StartServers 5

MinSpareServers 5

MaxSpareServers 10

MaxClients 150

MaxRequestsPerChild 0

StartServers 2

MinSpareThreads 25

MaxSpareThreads 75

ThreadLimit 64

ThreadsPerChild 25

MaxClients 150

MaxRequestsPerChild 0

StartServers 2

MinSpareThreads 25

MaxSpareThreads 75

ThreadLimit 64

ThreadsPerChild 25

MaxClients 150

MaxRequestsPerChild 0

User ${APACHE_RUN_USER}

Group ${APACHE_RUN_GROUP}

AccessFileName .htaccess

<Files ~ “^.ht”>

Order allow,deny

Deny from all

Satisfy all

DefaultType None

HostnameLookups Off

ErrorLog ${APACHE_LOG_DIR}/error.log

LogLevel warn

Include mods-enabled/*.load

Include mods-enabled/*.conf

LogFormat “%v:%p %h %l %u %t “%r” %>s %O “%{Referer}i” “%{User-Agent}i”” vhost_combined

LogFormat “%h %l %u %t “%r” %>s %O “%{Referer}i” “%{User-Agent}i”” combined

LogFormat “%h %l %u %t “%r” %>s %O” common

LogFormat “%{Referer}i -> %U” referer

LogFormat “%{User-agent}i” agent

ServerTokens Prod

ServerSignature Off

TraceEnable Off

NameVirtualHost *:8004

Listen 8004

<VirtualHost *:8004>

SSLProxyEngine On

ProxyPass https://originIPorHOSTNAME/

Order deny,allow

Deny from all

Allow from 127.0.0.1

ErrorLog ${APACHE_LOG_DIR}/error.log

LogLevel warn

CustomLog ${APACHE_LOG_DIR}/access.log combined