java comitted virtual memory problem

Hi, guys,

I’m having a problem related to the Java VM that Wowza runs in. I’ve modified the following in setenv.sh:

JAVA_OPTS="-Xmx4500M"
JAVA_OPTS="$JAVA_OPTS -server"
JAVA_OPTS="$JAVA_OPTS -XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode -XX:NewSize=1024m"

I have 6 GB RAM on the machine, I’m running CentOS 64bit. The Java VM grows in size until it ends up using 3-4 GB of swap and I can’t figure out why. Wowza is the only thing running on that server.

The Java VM summary from JConsole is (I’ve replaced the server address with [***] for security purposes):

VM Summary
Tuesday, April 20, 2010 11:45:10 AM EEST
 
Connection name:*
admin@service:jmx:rmi://[***]:8084/jndi/rmi://[***]:8085/jmxrmi
Virtual Machine:*
Java HotSpot(TM) 64-Bit Server VM version 14.3-b01
Vendor:*
Sun Microsystems Inc.
Name:*
22791@[***]
Uptime:*
7 hours 35 minutes
Process CPU time:*
10 hours 6 minutes
JIT compiler:*
HotSpot 64-Bit Server Compiler
Total compile time:*
1 minute
 
Live threads:*
***941
Peak:*
*1,025
Daemon threads:*
****59
Total threads started:*
98,653
Current classes loaded:*
3,105
Total classes loaded:*
3,120
Total classes unloaded:*
***15
 
Current heap size:*
1,522,709 kbytes
Maximum heap size:*
4,503,168 kbytes
Committed memory:*
4,503,168 kbytes
Pending finalization:*
0 objects
Garbage collector:*
Name = 'ParNew', Collections = 6,819, Total time spent = 33 minutes
Garbage collector:*
Name = 'ConcurrentMarkSweep', Collections = 4, Total time spent = 8 minutes
 
Operating System:*
Linux 2.6.18-164.11.1.el5
Architecture:*
amd64
Number of processors:*
4
Committed virtual memory:*
11,868,356 kbytes
Total physical memory:*
*6,108,868 kbytes
Free physical memory:*
****98,040 kbytes
Total swap space:*
12,550,136 kbytes
Free swap space:*
*9,883,132 kbytes
 
VM arguments:*
-Xmx4500M -XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode -XX:NewSize=1024m -Djava.net.preferIPv4Stack=true -Dcom.sun.management.jmxremote=true -Dcom.wowza.wms.AppHome=/usr/local/WowzaMediaServer -Dcom.wowza.wms.ConfigURL= -Dcom.wowza.wms.ConfigHome=/usr/local/WowzaMediaServer 
Class path:*
/usr/local/WowzaMediaServer/bin/wms-bootstrap.jar
Library path:*
/usr/java/jre1.6.0_17/lib/amd64/server:/usr/java/jre1.6.0_17/lib/amd64:/usr/java/jre1.6.0_17/../lib/amd64:/usr/java/packages/lib/amd64:/lib:/usr/lib
Boot class path:*
/usr/java/jre1.6.0_17/lib/resources.jar:/usr/java/jre1.6.0_17/lib/rt.jar:/usr/java/jre1.6.0_17/lib/sunrsasign.jar:/usr/java/jre1.6.0_17/lib/jsse.jar:/usr/java/jre1.6.0_17/lib/jce.jar:/usr/java/jre1.6.0_17/lib/charsets.jar:/usr/java/jre1.6.0_17/classes

So, from what I gather, the heap size is the right size, but the total virtual memory exceeds the heap size by more than twice it’s value. So if it isn’t heap size, what else is eating up all that RAM?

I really don’t know what would be causing this. I don’t understand how VM works at a detailed level in linux.

Charlie

In that case a RAID 0 or RAID 10 disk array will help.

Richard

I’m researching yet but I’m starting to think that Centos is not the best option for Wowza. I’ll post if I find out.

It wasn’t Centos finally (I had reasons to suspect) but now it’s free of all charges. It was the java version (1.6.0.18), that is buggy crap.

I’m researching yet but I’m starting to think that Centos is not the best option for Wowza. I’ll post if I find out.

Well, if you find out something, please share. Also, this problem is, from what I’ve noticed, related to vod, mostly, so disk access. A server with only live streams has no problems. Maybe you could also look into disk access schedulers and other things that might be related.

In that case a RAID 0 or RAID 10 disk array will help.

Richard

The server I was talking about has RAID 0, software I admit.

It wasn’t Centos finally (I had reasons to suspect) but now it’s free of all charges. It was the java version (1.6.0.18), that is buggy crap.

So updating with another Java version fixed it? What Java version did you update with?