wowza freezes

Wowza Streaming Engine
1

one of our servers recently started to have some kind of wowza freeze, it is not because there are many users on it, I cannot find any correlation that it goes down if there are a high number of viewers. It is happening sometimes 2 -3 times a day.

server: e5-2620 (24 cores)

RAM 64 giga

64 bit

centos

Wowza Media Server 3.6.3 build8031

below link has the wowzamediaserver_error.log, it will show the process going on and then at 21:56:11 when I restarted wowza.

http://hosting-marketers.com/wowzamediaserver_error.log

let me know if you can understand what is happening.

Thank you

2

just now it happen again:

root@magenta [~]# cd /usr/local/WowzaMediaServer/logs

root@magenta # tail -f wowzamediaserver_error.

log

java.lang.OutOfMemoryError: GC overhead limit exceeded

ERROR server comment 2014-03-02 06:14:51 - - - 1

7784.423 - - - - - - - - i

nvoke(onConnect): java.lang.OutOfMemoryError: GC overhead limit exceeded: unknow

n

WARN server comment 2014-03-02 06:14:56 - - - 1

7789.635 - - - - - - - - U

nexpected exception.: java.lang.OutOfMemoryError: GC overhead limit exceeded|

java.lang.OutOfMemoryError: GC overhead limit exceeded

WARN server comment 2014-03-02 06:17:06 - - - 1

7919.447 - - - - - - - - U

nexpected exception.: java.lang.OutOfMemoryError: GC overhead limit exceeded|

java.lang.OutOfMemoryError: GC overhead limit exceeded

WARN server comment 2014-03-02 06:17:07 - - - 1

7920.542 - - - - - - - - U

nexpected exception.: java.lang.OutOfMemoryError: GC overhead limit exceeded|

java.lang.OutOfMemoryError: GC overhead limit exceeded

WARN server comment 2014-03-02 06:18:15 - - - 1

7988.054 - - - - - - - - U

nexpected exception.: java.lang.OutOfMemoryError: GC overhead limit exceeded|

java.lang.OutOfMemoryError: GC overhead limit exceeded

3

we done that of course, but I noticed now that on /bin/setenv.sh instead of JAVA_OPTS="-Xmx8000M" I have:

JAVA_OPTS="-server -Xmx8000M"

? is it our mistake or this way is also correct?

4

I think I found the problem. I rechecked the tuning and I found that on [install-dir]/vhosts/default/conf/VHost.xml where I suppose to have:

RTP/DatagramConfiguration/UnicastIncoming/ProcessorCount: [total-core-count] (maximum of 12)

RTP/DatagramConfiguration/UnicastOutgoing/ProcessorCount: 2x[total-core-count] (maximum of 24)

RTP/DatagramConfiguration/MulticastIncoming/ProcessorCount: [total-core-count] (maximum of 12)

RTP/DatagramConfiguration/MulticastOutgoing/ProcessorCount: [total-core-count] (maximum of 12)

instead of 12 I had 24. since this was corrected the server is stable.

Thanks

5

it started again, I pasted the part of the wowzamediaserver_error.log which refers to this event:

http://pastebin.com/Sr6fKSxd

the server has 64 giga of ram, perhaps I should increase the JAVA_OPTS ram allocation?

another point, some of the errors there refer to services that have have been terminated and the app is no more inside the /conf directory anymore.

6

OK, done, email send to support@wowza.com

Thanks

7

Have you worked through the Performance Tuning Guide?

Salvadore

8

It looks like you are using the wrong class path for ModuleRTMPAuthenticate, for Wowza 3.5 and up the class path should be:

<Module>
	<Name>ModuleRTMPAuthenticate</Name>
	<Description>ModuleRTMPAuthenticate</Description>
	<Class>com.wowza.wms.security.ModuleRTMPAuthenticate</Class>
</Module>

This is probably not the issue, but will clean up the logs and fix the error as well as enabling the module.

Also, if you are not using Smooth Streaming you should disable it any other LiveStreamPacketizers you are not using.

Do you have any custom modules enabled in the application?

What is the load like on the server?

What type of streaming are you doing and what is the source?

Once you have made the recommended changes, please restart Wowza in stand-alone mode [install-dir]/bin/startup.bat and post the start up messages from the console so we can take a look.

Salvadore

9

Thanks for the update, glad to hear you have it sorted out.

Salvadore

10

I think it may be best if you open a ticket. If at all possible you should restart the server, delete the old logs and when the issue arises again zip up the /logs and /conf folders send to support@wowza.com.

If this is not possible, please open a ticket by zipping your /logs and /conf folders and send them to support@wowza.com. Also include a link to this thread as reference. Provide as much detail about your setup as possible, as well as any custom modules you may have implemented.

Salvadore