Max CCU for server E5-1620, 8G RAM, 8 HDD Raid 10

Duc_Thuan · March 11, 2013, 5:07pm

Hi all,

My server info:

CPU: E5-1620

RAM: 8G

8HDD x 3T Raid 10 (Seagate® Constellation™ ES.2 SATA 6Gb/s 7200rpm 64MB 3.0TB)

uplink: 4G

how many ccu can use on my server, please help me Benchmark Performance.

thanks.

Jason_Hilton · March 12, 2013, 11:09am

Hi

For a “live” stream with a bitrate of 500 kbps I estimate an absolute maximum of 8000 concurrent users based on the network but if you’re bonding GigE’s together maybe 7000.

For “on demand” streaming which I assume will be the main use of this server, it’s hard to say as you would need to speed test the array with load.

You can get the Wowza load test tool which supports on demand load testing from here,

https://www.wowza.com/docs/how-to-get-flash-rtmp-load-test-tool

The server will also need to be tuned and a tuning guide can be found here,

https://www.wowza.com/docs/how-to-do-performance-tuning

Jason

Richard_Lanham · March 14, 2013, 12:07pm

It’s probably much more than the load test tool can produce. You might have to use 3 or 4 load test clients

Richard

Richard_Lanham · March 14, 2013, 2:29pm

Correction, I meant it is more than the client (server you are running the load test client on) and/or network can handle. It is possible that it is the limit of the test server’s network bandwidth.

Richard

Richard_Lanham · March 19, 2013, 5:05pm

It looks like you have tuned for 12 cores. Do you have 12 core machine?

Do you have enough available memory to allocate 8000M java heap size?

Show the most recent startup lines from your access log, from the last time you re-started Wowza.

Richard

Richard_Lanham · March 20, 2013, 11:45pm

This line shows there is very little available memory when you started Wowza:

Hardware Physical Memory: 764MB/15927MB

The number on the left is available memory, on the right it is total memory. So there might be a lot of other activity on this server, other services.

Richard

Richard_Lanham · March 21, 2013, 3:06pm

Also, since Wowza/Java reserves memory (java heap size setting) you are not getting an accurate picture this way. Use JConsole instead:

https://www.wowza.com/docs/quick-start-guides

Richard

Richard_Lanham · March 21, 2013, 3:08pm

Corrected link is: https://www.wowza.com/docs/quick-start-guides

Jason_Hilton · March 21, 2013, 2:57pm

Hi,

According to the startup log as Richard said Wowza thinks that you only have 764MB free.

You should be looking at trying to make the amount of memory that you have available accessible by Wowza.

I know that you have tuned the server to take 8000MB memory but Wowza can’t see that its free for some reason and this need to be sorted first.

Jason

Jason_Hilton · March 25, 2013, 7:37pm

Hi

Looking closer at you VHost.xml your NetConnections / ProcessorCount is:

<ProcessorCount>16</ProcessorCount>

To keep to the calculated core count it should be:

<ProcessorCount>24</ProcessorCount>

Jason

Duc_Thuan · March 14, 2013, 10:29am

Hi Jason,

I use loadtool with info:

workerCount

8000

fileCount

10

connectionString

vod/definst

Video file size: 500MB

Video bitrate: 700kpbs

But traffic network only up to 2Mpbs, please help me config loadtool 2.2.4

thanks

Duc_Thuan · March 15, 2013, 8:10am

Hi Richard,

thank you very much.

Duc_Thuan · March 19, 2013, 9:34am

Hi Richard,

My server used NIC bonding (1 + 1) with 2G bw, I have download files test bw up to 1.9G, but when i used WowzaMediaServer loadtool with: 1200 ccu and 1.3G bw, java had die, wowza die, please help me config.

my file config info:

VHost.xml

[HTML]

24

*

1935

true

120000

true

100

cupertinostreaming,smoothstreaming

com.wowza.wms.http.HTTPCrossdomain

*crossdomain.xml

none

com.wowza.wms.http.HTTPClientAccessPolicy

*clientaccesspolicy.xml

none

com.wowza.wms.http.HTTPServerVersion

*

admin-digest

1

*

8086

true

16000

true

100

com.wowza.wms.http.streammanager.HTTPStreamManager

streammanager*

admin-digest

com.wowza.wms.http.HTTPServerInfoXML

serverinfo*

admin-digest

com.wowza.wms.http.HTTPConnectionInfo

connectioninfo*

admin-digest

com.wowza.wms.http.HTTPConnectionCountsXML

connectioncounts*

admin-digest

com.wowza.wms.http.HTTPServerVersion

*

none

smoothstreaming

cupertinostreaming

480

320

24

50

5

16

250

true

65000

24000

true

100

2000

90000

250

75

true

256000

65000

50

4096

true

65000

50

4096

12

24

12

60000

12000

20000

0

60000

true

[/HTML]

wms.sh

[HTML]#!/bin/sh

NOTE: Here you can configure the JVM’s built in JMX interface.

See the “Server Management Console and Monitoring” chapter

of the “User’s Guide” for more information on how to configure the

remote JMX interface in the [install-dir]/conf/Server.xml file.

JMXOPTIONS=-Dcom.sun.management.jmxremote=true

if [ “$WMSCOMMAND” != “start” ]; then

JMXOPTIONS=""

fi

ulimit -n unlimited

log interceptor com.wowza.wms.logging.LogNotify - see Javadocs for ILogNotify

$_EXECJAVA $JAVA_OPTS $JMXOPTIONS -Dcom.wowza.wms.AppHome="$WMSAPP_HOME" -Dcom.wowza.wms.ConfigURL="$WMSCONFIG_URL" -Dcom.wowza.wms.ConfigHome="$WMSCONFIG_HOME" -cp $WMSAPP_HOME/bin/wms-bootstrap.jar com.wowza.wms.bootstrap.Bootstrap $WMSCOMMAND >/dev/null 2>&1 &

if [ “$WMSCOMMAND” = “start” ]; then

echo $! > ${WMSPIDFILE}

wait $!

fi

exit 0

[/HTML]

setenv.sh

[HTML]#!/bin/sh

_EXECJAVA=java

#JAVA_OPTS="-Xmx1200M"

JAVA_OPTS="-Xmx8000M"

Uncomment to run server environment (faster), Note: will only work if server VM install, comes with JDL

JAVA_OPTS="$JAVA_OPTS -server"

Can be a better GC setting to avoid long pauses

#JAVA_OPTS="$JAVA_OPTS -XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode -XX:+UseParNewGC"

#JAVA_OPTS="$JAVA_OPTS -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:NewSize=512m"

Uncomment to fix multicast crosstalk problem when streams share multicast port

#JAVA_OPTS="$JAVA_OPTS -Djava.net.preferIPv4Stack=true"

WMSAPP_HOME=/usr/local/WowzaMediaServer

WMSCONFIG_HOME=/usr/local/WowzaMediaServer

WMSCONFIG_URL=

export WMSAPP_HOME WMSCONFIG_HOME JAVA_OPTS _EXECJAVA

[/HTML]

Duc_Thuan · March 20, 2013, 1:36pm

Hi Richard,

My server have 6 core thread 12

I have change my server info:

CPU: E5-2620

RAM: 16G

8HDD x 3T Raid 10 (Seagate® Constellation™ ES.2 SATA 6Gb/s 7200rpm 64MB 3.0TB)

uplink: 2G (bonding mode 5)

WowzaMediaServer_access_log

[HTML]#Version: 1.0

#Start-Date: 2013-03-18 15:27:24 ICT

#Software: Wowza Media Server 2.2.4 build27452

#Date: 2013-03-18

#Fields: date time tz x-event x-category x-severity x-status x-ctx x-comment x-vhost x-app x-appinst x-duration s-ip s-port s-uri c-ip c-proto c-referrer c-user-agent c-client-id cs-bytes sc-bytes x-stream-id x-spos cs-stream-bytes sc-stream-bytes x-sname x-sname-query x-file-name x-file-ext x-file-size x-file-length x-suri x-suri-stem x-suri-query cs-uri-stem cs-uri-query

2013-03-18 15:27:25 ICT server-start server INFO 200 Wowza Media Server 2 Perpetual 2.2.4 build27452 - - - - 0.918 - - - - - - - - - - - - - - - - - - - - - - - - -

2013-03-18 15:27:25 ICT comment server INFO 200 - Serial number: XXXXX-XXXXX-XXXXX-XXXXX-***** - - - 0.92 - - - - - - - - - - - - - - - - - - - - - - - - -

2013-03-18 15:27:25 ICT comment server INFO 200 - Maximum connections: Unlimited - - - 0.921 - - - - - - - - - - - - - - - - - - - - - - - - -

2013-03-18 15:27:25 ICT comment server INFO 200 - Hardware Available Processors: 12 - - - 0.922 - - - - - - - - - - - - - - - - - - - - - - - - -

2013-03-18 15:27:25 ICT comment server INFO 200 - Hardware Physical Memory: 764MB/15927MB - - - 0.923 - - - - - - - - - - - - - - - - - - - - - - - - -

2013-03-18 15:27:25 ICT comment server INFO 200 - Hardware Swap Space: 15990MB/15999MB - - - 0.924 - - - - - - - - - - - - - - - - - - - - - - - - -

2013-03-18 15:27:25 ICT comment server INFO 200 - Max File Descriptor Count: 4096 - - - 0.925 - - - - - - - - - - - - - - - - - - - - - - - - -

2013-03-18 15:27:25 ICT comment server INFO 200 - Open File Descriptor Count: 45 - - - 0.926 - - - - - - - - - - - - - - - - - - - - - - - - -

2013-03-18 15:27:25 ICT comment server INFO 200 - OS Name: Linux - - - 0.926 - - - - - - - - - - - - - - - - - - - - - - - - -

2013-03-18 15:27:25 ICT comment server INFO 200 - OS Version: 2.6.32-358.el6.x86_64 - - - 0.927 - - - - - - - - - - - - - - - - - - - - - - - - -

2013-03-18 15:27:25 ICT comment server INFO 200 - OS Architecture: amd64 - - - 0.927 - - - - - - - - - - - - - - - - - - - - - - - - -2013-03-18 15:27:25 ICT comment server INFO 200 - Java Name: Java HotSpot™ 64-Bit Server VM - - - 0.928 - - - - - - - - - - - - - - - - - - - - - - - - -2013-03-18 15:27:25 ICT comment server INFO 200 - Java Vendor: Oracle Corporation - - - 0.929 - - - - - - - - - - - - - - - - - - - - - - - - -2013-03-18 15:27:25 ICT comment server INFO 200 - Java Version: 1.7.0 - - - 0.929 - - - - - - - - - - - - - - - - - - - - - - - - -2013-03-18 15:27:25 ICT comment server INFO 200 - Java VM Version: 21.0-b17 - - - 0.93 - - - - - - - - - - - - - - - - - - - - - - - - -2013-03-18 15:27:25 ICT comment server INFO 200 - Java Spec Version: 1.7 - - - 0.93 - - - - - - - - - - - - - - - - - - - - - - - - -2013-03-18 15:27:25 ICT comment server INFO 200 - Java Home: /usr/java/jdk1.7.0/jre - - - 0.93 - - - - - - - - - - - - - - - - - - - - - - - - -2013-03-18 15:27:25 ICT comment server INFO 200 - Java Max Heap Size: 7111MB - - - 0.931 - - - - - - - - - - - - - - - - - - - - - - - - -2013-03-18 15:27:25 ICT comment server INFO 200 - Java Architecture: 64 - - - 0.931 - - - - - - - - - - - - - - - - - - - - - - - - -2013-03-18 15:27:25 ICT comment server INFO 200 - Java Encoding[file.encoding]: UTF-8 - - - 0.932 - - - - - - - - - - - - - - - - - - - - - - - - -2013-03-18 15:27:25 ICT comment server INFO 200 - Java Args[0]: -Xmx8000M - - - 0.932 - - - - - - - - - - - - - - - - - - - - - - - - -2013-03-18 15:27:25 ICT comment server INFO 200 - Java Args[1]: -Djava.net.preferIPv4Stack=true - - - 0.933 - - - - - - - - - - - - - - - - - - - - - - - - -2013-03-18 15:27:25 ICT comment server INFO 200 - Java Args[2]: -Dcom.sun.management.jmxremote=true - - - 0.933 - - - - - - - - - - - - - - - - - - - - - - - - -2013-03-18 15:27:25 ICT comment server INFO 200 - Java Args[3]: -Dcom.wowza.wms.AppHome=/usr/local/WowzaMediaServer - - - 0.933 - - - - - - - - - - - - - - - - - - - - - - - - -2013-03-18 15:27:25 ICT comment server INFO 200 - Java Args[4]: -Dcom.wowza.wms.ConfigURL= - - - 0.934 - - - - - - - - - - - - - - - - - - - - - - - - -2013-03-18 15:27:25 ICT comment server INFO 200 - Java Args[5]: -Dcom.wowza.wms.ConfigHome=/usr/local/WowzaMediaServer - - - 0.934 - - - - - - - - - - - - - - - - - - - - - - - - -2013-03-18 15:27:25 ICT comment server INFO 200 - Java GC[0]: PS Scavenge - - - 0.935 - - - - - - - - - - - - - - - - - - - - - - - - -2013-03-18 15:27:25 ICT comment server INFO 200 - Java GC[1]: PS MarkSweep - - - 0.936 - - - - - - - - - - - - - - - - - - - - - - - - -2013-03-18 15:27:25 ICT comment server INFO 200 - Server threads[h/t]: 10/10 - - - 0.936 - - - - - - - - - - - - - - - - - - - - - - - - -2013-03-18 15:27:26 ICT comment server INFO 200 - CMDInterface now listening: [any]:8083 - - - 2.698 - - - - - - - - - - - - - - - - - - - - - - - - -2013-03-18 15:27:27 ICT vhost-start vhost INFO 200 defaultVHost - - - - 3.374 - - - - - - - - - - - - - - - - - - - - - - - - -2013-03-18 15:27:27 ICT comment server INFO 200 - defaultVHost threads[h/t]:480/320 home:/usr/local/WowzaMediaServer - - - 3.439 - - - - - - - - - - - - - - - - - - - - - - - - -2013-03-18 15:27:27 ICT comment vhost INFO 200 defaultVHost Bind attempt ([any]:1935:16) - - - 3.443 - - - - -

- - - - -2013-03-18 15:27:27 ICT comment vhost INFO 200 defaultVHost Bind successful ([any]:1935) - - - 3.446 - - - - - - - - - - - - - - - - - - - - - - - - -2013-03-18 15:27:27 ICT comment vhost INFO 200 defaultVHost Bind attempt ([any]:8086:1) - - - 3.448 - - - - - - - - - - - - - - - - - - - - - - - - -2013-03-18 15:27:27 ICT comment vhost INFO 200 defaultVHost Bind successful ([any]:8086) - - - 3.452 - - - - - - - - - - - - - - - - - - - - - - - - -2013-03-18 15:27:27 ICT comment server INFO 200 - Wowza Media Server is started! - - - 3.458 - - - - - - - - - - - - - - - - - - - - - - - - -2013-03-18 15:27:29 ICT comment server WARN 200 - ApplicationInstance.setHTTPStreamerList[MyStream_film/definst]: HTTPStreamer is not valid: sanjosestreaming - - - 5.257 - - - - - - - - - - - - - - - - - - - - - - - - -2013-03-18 15:27:29 ICT comment server INFO 200 - SecureTokenTarget: create:true play:false publish:false - - - 5.311 - - - - - - - - - - - - - - - - - - - - - - - - -2013-03-18 15:27:29 ICT app-start application INFO 200 definst MyStream_film/definst - - - 5.356 - - - - - - - - - - - - - - - - -

[/HTML]

Duc_Thuan · March 21, 2013, 9:17am

Wowza streaming with 300 ccu, Memory info when used command free mem

[HTML][root@localhost ~]# free mem

total used free shared buffers cached

Mem: 16309900 16135160 174740 0 4984 13553440

-/+ buffers/cache: 2576736 13733164

Swap: 16383992 187328 16196664

[/HTML]

i think memory available 13733164

Duc_Thuan · March 25, 2013, 9:11am

When 1500 ccu online, my server was die and report error:

[HTML]ERROR server comment 2013-03-23 22:29:38 - - - - - 452

376.213 - - - - - - - - MediaReaderH264Cupertino.fi

llChunk: java.lang.NullPointerException

java.lang.NullPointerException

at com.wowza.wms.server.ServerHandler.exceptionCaught(Unknown Source)

at com.wowza.wms.server.ServerHandlerThreaded.exceptionCaught(Unknown Source)

at org.apache.mina.common.support.AbstractIoFilterChain$2.exceptionCaught(Unknown Source)

at org.apache.mina.common.support.AbstractIoFilterChain.callNextExceptionCaught(Unknown Sou

rce)

at org.apache.mina.common.support.AbstractIoFilterChain.access$800(Unknown Source)

at org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.exceptionCaught(Unknown Source)

at org.apache.mina.common.IoFilterAdapter.exceptionCaught(Unknown Source)

at org.apache.mina.common.support.AbstractIoFilterChain.callNextExceptionCaught(Unknown Source)

at org.apache.mina.common.support.AbstractIoFilterChain.access$800(Unknown Source)

at org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.exceptionCaught(Unknown Source)

at org.apache.mina.filter.executor.ExecutorFilter.processEvent(Unknown Source)

at org.apache.mina.filter.executor.ExecutorFilter$ProcessEventsRunnable.run(Unknown Source)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)

at java.lang.Thread.run(Thread.java:722)

ERROR server comment 2013-03-23 22:33:50 - - - - - 454528.93 - - - - - -

- - sessionOpened: java.lang.NullPointerException

ERROR server comment 2013-03-23 22:33:50 - - - - - 454528.979 - - - - - -

- - Application.doWatchdog: checkClientTimeout: java.lang.NullPointerException
- - 2013-03-23 22:33:50 - - - - - 454528.93 - - - - - -

java.lang.NullPointerException

at com.wowza.wms.server.ServerHandler.exceptionCaught(Unknown Source)

at com.wowza.wms.server.ServerHandlerThreaded.exceptionCaught(Unknown Source)

at org.apache.mina.common.support.AbstractIoFilterChain$2.exceptionCaught(Unknown Source)

at org.apache.mina.common.support.AbstractIoFilterChain.callNextExceptionCaught(Unknown Source)

at org.apache.mina.common.support.AbstractIoFilterChain.access$800(Unknown Source)

at org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.exceptionCaught(Unknown Source)

at org.apache.mina.common.support.AbstractIoFilterChain$1.exceptionCaught(Unknown Source)

at org.apache.mina.common.support.AbstractIoFilterChain.callNextExceptionCaught(Unknown Source)

at org.apache.mina.common.support.AbstractIoFilterChain.fireExceptionCaught(Unknown Source)

at org.apache.mina.common.support.AbstractIoFilterChain.callNextSessionClosed(Unknown Source)

at org.apache.mina.common.support.AbstractIoFilterChain.access$600(Unknown Source)

at org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.sessionClosed(Unknown Source)

at org.apache.mina.filter.codec.ProtocolCodecFilter.sessionClosed(Unknown Source)

at org.apache.mina.common.support.AbstractIoFilterChain.callNextSessionClosed(Unknown Source)

at org.apache.mina.common.support.AbstractIoFilterChain.access$600(Unknown Source)

at org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.sessionClosed(Unknown Source)

at org.apache.mina.filter.executor.ExecutorFilter.processEvent(Unknown Source)

at org.apache.mina.filter.executor.ExecutorFilter$ProcessEventsRunnable.run(Unknown Source)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)

at java.lang.Thread.run(Thread.java:722)

- - 2013-03-23 22:33:50 - - - - - 454528.93 - - - - - -