Possible memory leak?

Wowza Streaming Engine
1

Hi,

sometimes we experience a Java.OutOfMemoryException in our wowza server and we don’t understand how could this happen. A short summary:

  • developer license ( no more than 10 connections ), most of the time < 5 concurrent connections

  • tuning already done following linux instructions

  • 1200M heap size

Analyzing a memory dump obtained with -XX:+HeapDumpOnOutOfMemoryError we see this strange object histogram.

43996 instances of class org.apache.mina.common.SimpleByteBufferAllocator$SimpleByteBuffer
43995 instances of class com.wowza.wms.server.RtmpRequestMessage
43993 instances of class com.wowza.wms.server.ServerHandlerEvent

every “SimpleByteBuffer” contains a byte[] with a size of 24000 (exactly the same size of the receiver buffer configured in VHost.xml).

every “RtmpRequestMessage” contains a “RtmpSessionInfo” object that refer to the same clientId.

Any hints? Charlie?

2

Zip up and send us your conf and logs folders and describe in detail your hardware and OS setup (support@wowza.com). Be sure to reference this thread.

Charlie

3

Have you followed the tuning guide?:

https://www.wowza.com/docs/how-to-do-performance-tuning

Richard

4

Yes, Wowza, streaming, uses a lot of everything.

Richard

5

I did look at the support side emails regarding the post you are asking about, it involved several custom modules that included audio processing. There wasn’t anything that seemed useful.

Richard

6

You can zip up and send us your conf and logs folders and describe in detail your hardware and OS setup (support@wowza.com). Be sure to reference this thread.

Richard

7

Did you ever hear back from rocco? What was the cause of this? I’m experiencing what appears to be the same issue (out of memory, enabled heap dump on OOME, saw 32,016 ServerHandlerEvents, each contain a RtmpRequestMessage, which each contain the SimpleByteBuffer, almost all of which are 16KB)

768MB heap, and we do have a larger number of users, perhaps 40 or 50 at the time, but most of them are only sending text/object data back and forth, not audio/video, so bandwidth is not an issue. Strangely it doesn’t seem to be directly related to number of users, before I’ve had over 100 concurrent connections without hitting the heap limit.

Nothing descriptive in the server logs that points out the problem, besides out of memory exceptions. I can send you the heap dump if you wish to analyze it directly.

8

Yes, I’ve gone through it. I was just wondering if there was any further information on this exact issue this person seemed to have.

I don’t have an entire server dedicated to wowza, and that server has only 4GB of ram, so I can’t really give it much more than what I’ve already given it. Perhaps Wowza just tends to be more memory hungry?

9

Well I’m not doing a lot of streaming, some of the users have webcams and mics, but mostly it’s data events. Before we rewrote to use AS3 & RMTP, we ran a different java server in 256MB without any issues.

Anyway, please let me know if you have any more details at all about this old case.