Hi,
It’s a really weird problem, and we’re not able to reproduce it so far.
Couple days ago, we found that after a time point, let’s say, 20:00, all http session destroy events were not logged.
It seemed many http sessions started without ended.
According to log, our custom module on wowza experienced an IO exception (Too many open files) that day, about six hours earlier than this weird problem happened.
java.io.IOException: Too many open files
at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
at org.apache.mina.transport.socket.nio.SocketAcceptor$Worker.processSessions(Unknown Source)
at org.apache.mina.transport.socket.nio.SocketAcceptor$Worker.run(Unknown Source)
at org.apache.mina.util.NamePreservingRunnable.run(Unknown Source)
at java.lang.Thread.run(Thread.java:722)
Could the exception be the cause?
We would want to know if there’s any chance that the timeout-timer for http session crashes and therefore prevents http sessions from being destroyed because of some programmatic error?
Thanks in advance,
weichung
Take a look at the ulimit settings in the Tuning Guide:
https://www.wowza.com/docs/how-to-do-performance-tuning
Richard
Why 65335? That happens to be the number of UDP ports we recommend opening for RTSP streaming. The tuning guide recommends setting ulimit to 20000 (twenty thousand)
Richard
I suppose you can try to raise the ulimit setting. I don’t think the “http session timeout-timer crashed”. It is very busy server, I take it.
Richard
Zip up and send /conf and /logs folders to support@wowza.com. Tell us which log file to look at, when did this happen? Have you changed any configuration since then? If so, please detail those changes so we know the difference between the logs we are looking at the configuration you are sending.
Include a link to this thread for reference.
Richard
Hi Richard,
Thanks for the quick reply!
Are you suggesting that it is the file open problem which overloads the server and thus no http session ends?
Cause we thought so at the first place, but not quite sure.
We have set the ulimit to be 65535 and temporarily fixed the problem, is this a good idea?
On the other hand, we also have checked our custom module and refined the file open process.
weichung
It seems that I kind of put the cart before the horse.
We guess that the cause might be the crash of http session timeout-timer, and from that moment no http session ever ends.
server log said that wowza exceeded the 20000 file-open limit and crashed a day and half later.
If we’re right, why would the timer fail?
weichung