Memory increase over time

Wowza Streaming Engine
1

We are trying to figure out why our Wowza servers memory usage increases until it finally maxes out.

Scenerio

======

We will eventually have nearly 1000 clients connecting to our 2, load balanced, Wowza servers for an IPTV setup that includes live streaming with DVR and vod capability. Right now, in testing, we don’t even have 50 clients connected at once.

Problem

======

After a few days, the Java heap size maxes out and the streams will start to occasionally skip. By monitoring JConsole, I noticed that the MemoryPool “CMS Old gen” gradually increases until ‘Committed Memory’ hits ‘Max’ (normally about 2 days), at which point all the other MemoryPools stop working correctly and we start to see errors with the streams (frames being dropped, stutter in video).

Setup

====

Encoders

(18) MakitoX2

(36) streams

Video Out: 720p

FPS: 60

GOP: 120

UDP Port: 10000-10500

Bitrate: 3000

Servers

Server: (2)HP DL380G8

CPU: (2)XeonE5(8core, 32 threads)

RAM: 32GB

NIC: (4)1Gb Bonded mode 6

Harddrive: (5)146GB 6G SAS 15k RPM RAID10

OS: Ubuntu 12.04.3

Java: jdk 1.7.0_45

Wowza Media Server: 3.6.4

Wowza Settings

VHost.xml

<Root>
        <VHost>
                <HostPortList>
                        <HostPort>
                                <ProcessorCount>24</ProcessorCount>
                                <IpAddress>*</IpAddress>
                                <!-- Separate multiple ports with commas -->
                                <!-- 80: HTTP, RTMPT -->
                                <!-- 554: RTSP -->
                                <Port>1935</Port>
                                <HTTPIdent2Response></HTTPIdent2Response>
                                <SocketConfiguration>
                                        <ReuseAddress>true</ReuseAddress>
                                        <!-- suggested settings for video on demand applications -->
                                        <ReceiveBufferSize>65000</ReceiveBufferSize>
                                        <ReadBufferSize>65000</ReadBufferSize>
                                        <SendBufferSize>65000</SendBufferSize>
                                        <!-- suggest settings for low latency chat and video recording applications
                                        <ReceiveBufferSize>32000</ReceiveBufferSize>
                                        <ReadBufferSize>32000</ReadBufferSize>
                                        <SendBufferSize>32000</SendBufferSize>
                                        -->
                                        <KeepAlive>true</KeepAlive>
                                        <!-- <TrafficClass>0</TrafficClass> -->
                                        <!-- <OobInline>false</OobInline> -->
                                        <!-- <SoLingerTime>-1</SoLingerTime> -->
                                        <!-- <TcpNoDelay>false</TcpNoDelay> -->
                                        <AcceptorBackLog>100</AcceptorBackLog>
                                </SocketConfiguration>
                                <HTTPStreamerAdapterIDs>cupertinostreaming,smoothstreaming,sanjosestreaming,dvrchunkstreaming,mpegdashstreaming</HTTPStreamerAdapterIDs>
                                <HTTPProviders>
                                        <HTTPProvider>
                                                <BaseClass>com.wowza.wms.http.HTTPCrossdomain</BaseClass>
                                                <RequestFilters>*crossdomain.xml</RequestFilters>
                                                <AuthenticationMethod>none</AuthenticationMethod>
                                        </HTTPProvider>
                                        <HTTPProvider>
                                                <BaseClass>com.wowza.wms.http.HTTPClientAccessPolicy</BaseClass>
                                                <RequestFilters>*clientaccesspolicy.xml</RequestFilters>
                                                <AuthenticationMethod>none</AuthenticationMethod>
                                        </HTTPProvider>
                                        <HTTPProvider>
                                                <BaseClass>com.wowza.wms.http.HTTPProviderMediaList</BaseClass>
                                                <RequestFilters>*jwplayer.rss|*jwplayer.smil|*medialist.smil|*manifest-rtmp.f4m</RequestFilters>
                                                <AuthenticationMethod>none</AuthenticationMethod>
                                        </HTTPProvider>
                                        <!-- load balancer listener -->
                                        <HTTPProvider>
                                                <BaseClass>com.wowza.wms.plugin.loadbalancer.HTTPLoadBalancerRedirector</BaseClass>
                                                <RequestFilters>*loadbalancer</RequestFilters>
                                                <AuthenticationMethod>none</AuthenticationMethod>
                                                <Properties>
                                                        <Property>
                                                                <Name>enableServerInfoXML</Name>
                                                                <Value>true</Value>
                                                                <Type>Boolean</Type>
                                                        </Property>
                                                </Properties>
                                        </HTTPProvider>
                                        <HTTPProvider>
                                                <BaseClass>com.wowza.wms.http.HTTPServerVersion</BaseClass>
                                                <RequestFilters>*</RequestFilters>
                                                <AuthenticationMethod>none</AuthenticationMethod>
                                        </HTTPProvider>
                                </HTTPProviders>
                        </HostPort>
                        <!-- 443 with SSL -->
                        <!--
                        <HostPort>
                                <ProcessorCount>4</ProcessorCount>
                                <IpAddress>*</IpAddress>
                                <Port>443</Port>
                                <HTTPIdent2Response></HTTPIdent2Response>
                                <SSLConfig>
                                        <KeyStorePath>${com.wowza.wms.context.VHostConfigHome}/conf/keystore.jks</KeyStorePath>
                                        <KeyStorePassword>[password]</KeyStorePassword>
                                        <KeyStoreType>JKS</KeyStoreType>
                                        <SSLProtocol>TLS</SSLProtocol>
                                        <Algorithm>SunX509</Algorithm>
                                        <CipherSuites></CipherSuites>
                                        <Protocols></Protocols>
                                </SSLConfig>
                                <SocketConfiguration>
                                        <ReuseAddress>true</ReuseAddress>
                                        <ReceiveBufferSize>65000</ReceiveBufferSize>
                                        <ReadBufferSize>65000</ReadBufferSize>
                                        <SendBufferSize>65000</SendBufferSize>
                                        <KeepAlive>true</KeepAlive>
                                        <AcceptorBackLog>100</AcceptorBackLog>
                                </SocketConfiguration>
                                <HTTPStreamerAdapterIDs>cupertinostreaming,smoothstreaming,sanjosestreaming,dvrchunkstreaming,mpegdashstreaming</HTTPStreamerAdapterIDs>
                                <HTTPProviders>
                                        <HTTPProvider>
                                                <BaseClass>com.wowza.wms.http.HTTPCrossdomain</BaseClass>
                                                <RequestFilters>*crossdomain.xml</RequestFilters>
                                                <AuthenticationMethod>none</AuthenticationMethod>
                                        </HTTPProvider>
                                        <HTTPProvider>
                                                <BaseClass>com.wowza.wms.http.HTTPClientAccessPolicy</BaseClass>
                                                <RequestFilters>*clientaccesspolicy.xml</RequestFilters>
                                                <AuthenticationMethod>none</AuthenticationMethod>
                                        </HTTPProvider>
                                        <HTTPProvider>
                                                <BaseClass>com.wowza.wms.http.HTTPProviderMediaList</BaseClass>
                                                <RequestFilters>*jwplayer.rss|*jwplayer.smil|*medialist.smil|*manifest-rtmp.f4m</RequestFilters>
                                                <AuthenticationMethod>none</AuthenticationMethod>
                                        </HTTPProvider>
                                        <HTTPProvider>
                                                <BaseClass>com.wowza.wms.http.HTTPServerVersion</BaseClass>
                                                <RequestFilters>*</RequestFilters>
                                                <AuthenticationMethod>none</AuthenticationMethod>
                                        </HTTPProvider>
                                </HTTPProviders>
                        </HostPort>
                        -->
                        <!-- Admin HostPort -->
                        <HostPort>
                                <ProcessorCount>1</ProcessorCount>
                                <IpAddress>*</IpAddress>
                                <Port>8086</Port>
                                <HTTPIdent2Response></HTTPIdent2Response>
                                <SocketConfiguration>
                                        <ReuseAddress>true</ReuseAddress>
                                        <ReceiveBufferSize>16000</ReceiveBufferSize>
                                        <ReadBufferSize>16000</ReadBufferSize>
                                        <SendBufferSize>16000</SendBufferSize>
                                        <KeepAlive>true</KeepAlive>
                                        <AcceptorBackLog>100</AcceptorBackLog>
                                </SocketConfiguration>
                                <HTTPStreamerAdapterIDs></HTTPStreamerAdapterIDs>
                                <HTTPProviders>
                                        <HTTPProvider>
                                                <BaseClass>com.wowza.wms.http.streammanager.HTTPStreamManager</BaseClass>
                                                <RequestFilters>streammanager*</RequestFilters>
                                                <AuthenticationMethod>admin-digest</AuthenticationMethod>
                                        </HTTPProvider>
                                        <HTTPProvider>
                                                <BaseClass>com.wowza.wms.http.HTTPServerInfoXML</BaseClass>
                                                <RequestFilters>serverinfo*</RequestFilters>
                                                <AuthenticationMethod>admin-digest</AuthenticationMethod>
                                        </HTTPProvider>
                                        <HTTPProvider>
                                                <BaseClass>com.wowza.wms.http.HTTPConnectionInfo</BaseClass>
                                                <RequestFilters>connectioninfo*</RequestFilters>
                                                <AuthenticationMethod>admin-digest</AuthenticationMethod>
                                        </HTTPProvider>
                                        <HTTPProvider>
                                                <BaseClass>com.wowza.wms.http.HTTPConnectionCountsXML</BaseClass>
                                                <RequestFilters>connectioncounts*</RequestFilters>
                                                <AuthenticationMethod>admin-digest</AuthenticationMethod>
                                        </HTTPProvider>
                                        <HTTPProvider>
                                                <BaseClass>com.wowza.wms.transcoder.httpprovider.HTTPTranscoderThumbnail</BaseClass>
                                                <RequestFilters>transcoderthumbnail*</RequestFilters>
                                                <AuthenticationMethod>admin-digest</AuthenticationMethod>
                                        </HTTPProvider>
                                        <HTTPProvider>
                                                <BaseClass>com.wowza.wms.http.HTTPProviderMediaList</BaseClass>
                                                <RequestFilters>medialist*</RequestFilters>
                                                <AuthenticationMethod>admin-digest</AuthenticationMethod>
                                        </HTTPProvider>
                                        <HTTPProvider>
                                                <BaseClass>com.wowza.wms.livestreamrecord.http.HTTPLiveStreamRecord</BaseClass>
                                                <RequestFilters>livestreamrecord*</RequestFilters>
                                                <AuthenticationMethod>admin-digest</AuthenticationMethod>
                                        </HTTPProvider>
                                        <HTTPProvider>
                                                <BaseClass>com.wowza.wms.http.HTTPServerVersion</BaseClass>
                                                <RequestFilters>*</RequestFilters>
                                                <AuthenticationMethod>none</AuthenticationMethod>
                                        </HTTPProvider>
                                </HTTPProviders>
                        </HostPort>
                </HostPortList>
                <HTTPStreamerAdapters>
                        <HTTPStreamerAdapter>
                                <ID>smoothstreaming</ID>
                                <Name>smoothstreaming</Name>
                                <Properties>
                                </Properties>
                        </HTTPStreamerAdapter>
                        <HTTPStreamerAdapter>
                                <ID>cupertinostreaming</ID>
                                <Name>cupertinostreaming</Name>
                                <Properties>
                                </Properties>
                        </HTTPStreamerAdapter>
                        <HTTPStreamerAdapter>
                                <ID>sanjosestreaming</ID>
                                <Name>sanjosestreaming</Name>
                                <Properties>
                                </Properties>
                        </HTTPStreamerAdapter>
                        <HTTPStreamerAdapter>
                                <ID>dvrchunkstreaming</ID>
                                <Name>dvrchunkstreaming</Name>
                                <Properties>
                                </Properties>
                        </HTTPStreamerAdapter>
                        <HTTPStreamerAdapter>
                                <ID>mpegdashstreaming</ID>
                                <Name>mpegdashstreaming</Name>
                                <Properties>
                                </Properties>
                        </HTTPStreamerAdapter>
                </HTTPStreamerAdapters>
                <HandlerThreadPool>
                        <PoolSize>480</PoolSize>
                </HandlerThreadPool>
                <TransportThreadPool>
                        <PoolSize>320</PoolSize>
                </TransportThreadPool>
                <IdleWorkers>
                        <WorkerCount>24</WorkerCount>
                        <CheckFrequency>100</CheckFrequency>
                        <MinimumWaitTime>5</MinimumWaitTime>
                </IdleWorkers>
                <NetConnections>
                        <ProcessorCount>24</ProcessorCount>
                        <IdleFrequency>250</IdleFrequency>
                        <SocketConfiguration>
                                <ReuseAddress>true</ReuseAddress>
                                <ReceiveBufferSize>65000</ReceiveBufferSize>
                                <ReadBufferSize>65000</ReadBufferSize>
                                <SendBufferSize>65000</SendBufferSize>
                                <KeepAlive>true</KeepAlive>
                                <!-- <TrafficClass>0</TrafficClass> -->
                                <!-- <OobInline>false</OobInline> -->
                                <!-- <SoLingerTime>-1</SoLingerTime> -->
                                <!-- <TcpNoDelay>false</TcpNoDelay> -->
                                <AcceptorBackLog>100</AcceptorBackLog>
                        </SocketConfiguration>
                </NetConnections>
                <HTTPTunnel>
                        <KeepAliveTimeout>2000</KeepAliveTimeout>
                </HTTPTunnel>
                <Client>
                        <ClientTimeout>90000</ClientTimeout>
                        <IdleFrequency>500</IdleFrequency>
                </Client>
                <!-- RTP/Authentication/Methods defined in Authentication.xml. Default setup includes; none, basic, digest -->
                <RTP>
                        <IdleFrequency>75</IdleFrequency>
                        <DatagramConfiguration>
                                <Incoming>
                                        <ReuseAddress>true</ReuseAddress>
                                        <ReceiveBufferSize>1024000</ReceiveBufferSize>
                                        <SendBufferSize>65000</SendBufferSize>
                                        <!-- <MulticastBindToAddress>true</MulticastBindToAddress> -->
                                        <!-- <MulticastInterfaceAddress>192.168.1.22</MulticastInterfaceAddress> -->
                                        <!-- <TrafficClass>0</TrafficClass> -->
                                        <MulticastTimeout>50</MulticastTimeout>
                                        <DatagramMaximumPacketSize>4096</DatagramMaximumPacketSize>
                                </Incoming>
                                <Outgoing>
                                        <ReuseAddress>true</ReuseAddress>
                                        <ReceiveBufferSize>65000</ReceiveBufferSize>
                                        <SendBufferSize>65000</SendBufferSize>
                                        <!-- <MulticastBindToAddress>true</MulticastBindToAddress> -->
                                        <!-- <MulticastInterfaceAddress>192.168.1.22</MulticastInterfaceAddress> -->
                                        <!-- <TrafficClass>0</TrafficClass> -->
                                        <MulticastTimeout>50</MulticastTimeout>
                                        <DatagramMaximumPacketSize>4096</DatagramMaximumPacketSize>
                                </Outgoing>
                        </DatagramConfiguration>
                        <UnicastIncoming>
                                <ProcessorCount>12</ProcessorCount>
                        </UnicastIncoming>
                        <UnicastOutgoing>
                                <ProcessorCount>24</ProcessorCount>
                        </UnicastOutgoing>
                        <MulticastIncoming>
                                <ProcessorCount>12</ProcessorCount>
                        </MulticastIncoming>
                        <MulticastOutgoing>
                                <ProcessorCount>12</ProcessorCount>
                        </MulticastOutgoing>
                </RTP>
                <Application>
                        <ApplicationTimeout>60000</ApplicationTimeout>
                        <PingTimeout>12000</PingTimeout>
                        <UnidentifiedSessionTimeout>30000</UnidentifiedSessionTimeout>
                        <ValidationFrequency>20000</ValidationFrequency>
                        <MaximumPendingWriteBytes>0</MaximumPendingWriteBytes>
                        <MaximumSetBufferTime>60000</MaximumSetBufferTime>
                </Application>
                <StartStartupStreams>true</StartStartupStreams>
                <!-- Properties defined here will be added to the IVHost.getProperties() collection -->
                <Properties>
                </Properties>
        </VHost>
</Root>

bin/setenv.sh

#!/bin/sh
_EXECJAVA=java
# Run server environment
JAVA_OPTS="$JAVA_OPTS -server"
# Max heap size (-Xmx<max> -Xms<min>)
JAVA_OPTS="$JAVA_OPTS -Xmx10000M"
# Custom garbage collection
#JAVA_OPTS="$JAVA_OPTS -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:NewRatio=3" 
# Better garbage collection setting to avoid long pauses
JAVA_OPTS="$JAVA_OPTS -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:NewSize=512m"
# Uncomment to log debug garbage collection information
NOW=$(date +"%Y-%m-%d-%H-%M")
JAVA_OPTS="$JAVA_OPTS -verbose:gc -Xloggc:"/var/log/wowza/gc_$NOW.log" -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintHeapAtGC -XX:+PrintGCApplicationConcurrentTime -XX:+PrintGCApplicationStoppedTime"
# Uncomment to write heap dump on OutOfMemoryError
#JAVA_OPTS="$JAVA_OPTS -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/usr/local/WowzaMediaServer"
# Uncomment to force Java to use specific language settings
#JAVA_OPTS="$JAVA_OPTS -Duser.language=en -Duser.country=US -Dfile.encoding=Cp1252"
# Uncomment to fix multicast crosstalk problem when streams share multicast port
JAVA_OPTS="$JAVA_OPTS -Djava.net.preferIPv4Stack=true"
WMSAPP_HOME=/usr/local/WowzaMediaServer
WMSCONFIG_HOME=/usr/local/WowzaMediaServer
WMSCONFIG_URL=
export WMSAPP_HOME WMSCONFIG_HOME JAVA_OPTS _EXECJAVA

Applications

vod

liveorigin

liveedge

What we have tried

==================

We have tuned the server as recommended here;

We have looked through everything we could find on tuning in the forums and have tried many different ways for GC in setenv.sh. Here are some posts that we looked into:

https://www.wowza.com/forums/showthread.php?24647-Best-Tunning-Help-Request

On some of the posts, users were experiancing similar issues but it was resolved with upgrading Wowza. We have done that, however, and are still getting the memory increase.

If you have any ideas or suggestions on how we can resolve this, it would be very helpful. We have a huge deployment coming up that will include 6 more servers with nearly the exact same setup. I need to get these 2 working first so that we have an idea of how to deploy the rest. Thanks!

2

Update. I just realized that the VHost.xml file wasn’t tuned. I had done it and double checked in the past, but I might have lost it during an upgrade. I will monitor with the new changes and post results. That’s probably it =(.

3

Hi,

In the above case improvement was seen by using the G1 (Garbage first) collector.

It’s certainly worth a try, and may help if you’re seeing exactly the same issue.

Daren

4

Please zip up your /conf and /logs folders and send them to support@wowza.com to open a ticket.

Please also include a link to this thread as reference.

Salvadore

5

I have verified the servers are properly tuned but we are still getting the same result. I updated my original post to show what the current tuning is. Any ideas as to why the memory increases?

I have noticed more specifically what happens with the garbage collection. It seems that the “Par Eden space” clears out completely when it does it’s GC. The graph over time is exactly the same. It’s “CMS Old Gen” that slowly builds up. When it hits it’s ceiling, the “Par Eden Space” and the “Par survivor space” max out as well and we notice video errors. It seems that the big GC that is supposed to clean out “CMS Old Gen” can’t get it all in the time alloted, or can’t clear it all for some other reason.

Thanks for your help!

6

Ok sent. Let me know if you need anything else. Thanks Salvadore!

Jordan

7

Hi everyone,

Just a quick update. We have been running these tuned settings for about 9 months now with complete stability. Here is our GC:

bin/setenv.sh

#!/bin/sh
_EXECJAVA=java
# Run server environment
JAVA_OPTS="$JAVA_OPTS -server"
# Max heap size (-Xmx<max> -Xms<min>)
JAVA_OPTS="$JAVA_OPTS -Xmx10000M -Xms10000M"
# G1GC settings
JAVA_OPTS="$JAVA_OPTS -XX:+UseG1GC -XX:MaxGCPauseMillis=100 -XX:InitiatingHeapOccupancyPercent=45"
# Better garbage collection setting to avoid long pauses
#JAVA_OPTS="$JAVA_OPTS -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:NewSize=512m"
# Uncomment to log debug G1GC information
#NOW=$(date +"%Y-%m-%d-%H-%M")
#JAVA_OPTS="$JAVA_OPTS -Xloggc:"/var/log/wowza/gc_$NOW.log" -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintGCDateStamps"
# Uncomment to log debug garbage collection information
#NOW=$(date +"%Y-%m-%d-%H-%M")
#JAVA_OPTS="$JAVA_OPTS -verbose:gc -Xloggc:"/var/log/wowza/gc_$NOW.log" -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintHeapAtGC -XX:+PrintGCApplicationConcurrentTime -XX:+PrintGCApplicationStoppedTime"
# Uncomment to write heap dump on OutOfMemoryError
#JAVA_OPTS="$JAVA_OPTS -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/usr/local/WowzaMediaServer"
# Uncomment to force Java to use specific language settings
#JAVA_OPTS="$JAVA_OPTS -Duser.language=en -Duser.country=US -Dfile.encoding=Cp1252"
# Uncomment to fix multicast crosstalk problem when streams share multicast port
JAVA_OPTS="$JAVA_OPTS -Djava.net.preferIPv4Stack=true"
WMSAPP_HOME=/usr/local/WowzaMediaServer
WMSCONFIG_HOME=/usr/local/WowzaMediaServer
WMSCONFIG_URL=
export WMSAPP_HOME WMSCONFIG_HOME JAVA_OPTS _EXECJAVA

Hope that helps!

Jordan

8

Hi cymaknova,

You’re right, that was for Wowza 3. We recently migrated to Wowza 4, and after a LOT of load testing, we find that the default GC, CMS, works best for us. I’m not sure if our load testing just got better or there was a change to Wowza 4 tuning, but G1GC won’t last as long as CMS.

One thing you can check is the GC logs that are enabled in ‘conf/Tune.xml’. You’ll need to educate yourself on how to read them, but a quick google search of ‘reading gc logs’ will give you lots of places to start. A site I found that had a lot of good articles is:

https://blogs.oracle.com/poonam/

Once you learn how to read the GC logs, you can see what’s going on. Hope that helps!

Jordan

9

bump

Any update? I am seeing similar trends on my wowza install where over 2-3 day the memory is filled up and our streams become unstable and jittery.

10

Hi everyone,

Just a quick update. We have been running these tuned settings for about 9 months now with complete stability. Here is our GC:

bin/setenv.sh

#!/bin/sh
_EXECJAVA=java
# Run server environment
JAVA_OPTS="$JAVA_OPTS -server"
# Max heap size (-Xmx<max> -Xms<min>)
JAVA_OPTS="$JAVA_OPTS -Xmx10000M -Xms10000M"
# G1GC settings
JAVA_OPTS="$JAVA_OPTS -XX:+UseG1GC -XX:MaxGCPauseMillis=100 -XX:InitiatingHeapOccupancyPercent=45"
# Better garbage collection setting to avoid long pauses
#JAVA_OPTS="$JAVA_OPTS -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:NewSize=512m"
# Uncomment to log debug G1GC information
#NOW=$(date +"%Y-%m-%d-%H-%M")
#JAVA_OPTS="$JAVA_OPTS -Xloggc:"/var/log/wowza/gc_$NOW.log" -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintGCDateStamps"
# Uncomment to log debug garbage collection information
#NOW=$(date +"%Y-%m-%d-%H-%M")
#JAVA_OPTS="$JAVA_OPTS -verbose:gc -Xloggc:"/var/log/wowza/gc_$NOW.log" -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintHeapAtGC -XX:+PrintGCApplicationConcurrentTime -XX:+PrintGCApplicationStoppedTime"
# Uncomment to write heap dump on OutOfMemoryError
#JAVA_OPTS="$JAVA_OPTS -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/usr/local/WowzaMediaServer"
# Uncomment to force Java to use specific language settings
#JAVA_OPTS="$JAVA_OPTS -Duser.language=en -Duser.country=US -Dfile.encoding=Cp1252"
# Uncomment to fix multicast crosstalk problem when streams share multicast port
JAVA_OPTS="$JAVA_OPTS -Djava.net.preferIPv4Stack=true"
WMSAPP_HOME=/usr/local/WowzaMediaServer
WMSCONFIG_HOME=/usr/local/WowzaMediaServer
WMSCONFIG_URL=
export WMSAPP_HOME WMSCONFIG_HOME JAVA_OPTS _EXECJAVA

Hope that helps!

Jordan

I assume this is version 3.x. How about for Streaming Engine v4.x? I have the same problem with memory usage.

I stream a live stream to the server and have the transcoder on. The server record all incoming stream and then send that to my S3 bucket. My memory get close to 80% by the time I finished a 2hr session live streaming and recording. I am using an amazon EC2 instance c3.xlarge. I know it is at the low end, but the memory usage is way too high and I want to find out why and how could I fix it.

Thanks!

11

I assume this is version 3.x. How about for Streaming Engine v4.x? I have the same problem with memory usage.

I stream a live stream to the server and have the transcoder on. The server record all incoming stream and then send that to my S3 bucket. My memory get close to 80% by the time I finished a 2hr session live streaming and recording. I am using an amazon EC2 instance c3.xlarge. I know it is at the low end, but the memory usage is way too high and I want to find out why and how could I fix it.

Thanks!

cymankova,

Did you ever get to a resolution on this. I am currently working with Wowza support on a similar issue and wondering what steps you took. Our servers are based at Rackspace, but have experienced ever increasing memory utilization over time. We were running on 4.0.6, and saw problems in both Total Memory and Wowza Heap (both in the red). Updating to 4.1.2 took care of the heap but Total Memory creeped up over time.

Thanks,

Bob