RTP Rebroadcast Issue

Wowza Streaming Engine
1

System Overview

Canon VB-C50i

Axis

EC2 Inst. - FFServer (Elastic IP)

EC2 Inst. - Wowza (Elastic IP)

The Canon is streaming at 3fps in MJPEG which is feed into ffmpeg for conversion to H.264. Wowza is configured to rebroadcast the ffserver stream.

Rebroadcasting the Axis H.264 stream works on all clients (RTSP, RTMP and iPhone). However, rebroadcasting the ffserver stream does not work reliably over RTSP. It is very hit and miss if an RTSP stream comes through. Streaming with VLC or a an android phone directly from ffserver woks fine. I would prefer to stream through Wowza as I have more control over the content. What RTSP settings should I be looking at to tune this?

Update: (I am new to this, so please excuse my ignorance)

This appears to be a bit rate issue. I had ffmpeg set at a target rate of 50k. I bumped this up to 200k and everything sorted itself out. I still think it should work at the lower bit rates. Is there something I can set to get the stream to just play whatever arrives? It works fine when streaming directly from ffserver so I suspect this can be set somewhere. I have appended a VLC dump from ffserver.

Update 2: If I set the ffmpeg output to 15FPS with a target bitrate of 200kbs the problems disappear. Even when only feeding ffmpeg with 5fps. This also clears up the 5+ second buffering in the Flash player. Any pointers would be greatly appreciated.

Here is a dump from VLC of when it was successful

main debug: Activated
main debug: rebuilding array of current - root Playlist
main debug: rebuild done - 0 items, index -1
main debug: looking for services probe module: 5 candidates
main debug: no services probe module matching "any" could be loaded
main debug: TIMER module_need() : 190.020 ms - Total 190.020 ms / 1 intvls (Avg 190.020 ms)
macosx debug: using Snow Leopard AR cookies
macosx debug: notification received in VLC with name VLCOSXGUIInit and object VLCEyeTVSupport
main debug: looking for services probe module: 5 candidates
main debug: no services probe module matching "any" could be loaded
main debug: TIMER module_need() : 5.526 ms - Total 5.526 ms / 1 intvls (Avg 5.526 ms)
main debug: adding item `rtsp://:<removed>)
main debug: rebuilding array of current - root Playlist
main debug: rebuild done - 1 items, index -1
main debug: processing request item rtsp://<removed>
node Playlist skip 0
main debug: resyncing on rtsp://<removed>
main debug: rtsp://<removed> is at 0
main debug: starting new item
main debug: creating new input thread
main debug: Creating an input for 'rtsp://<removed>'
main debug: thread (input) created at priority 22 (input/input.c:214)
main debug: thread started
main debug: using timeshift granularity of 50 MiB
main debug: using timeshift path '/tmp'
main debug: `rtsp://<removed>' gives access `rtsp' demux `' path `<removed>'
main debug: creating demux: access='rtsp' demux='' path='<removed>'
main debug: looking for access_demux module: 1 candidate
macosx debug: input has changed, refreshing interface
live555 debug: RTP subsession 'video/H264'
main debug: selecting program id=0
live555 debug: setup start: 0.000000 stop:0.000000
live555 debug: We have a timeout of 60 seconds
live555 debug: spawned timeout thread
live555 debug: play start: 0.000000 stop:0.000000
main debug: using access_demux module "live555"
main debug: TIMER module_need() : 1538.884 ms - Total 1538.884 ms / 1 intvls (Avg 1538.884 ms)
main debug: looking for decoder module: 31 candidates
avcodec debug: libavcodec initialized (interface 0x345e03)
avcodec debug: trying to use direct rendering
avcodec debug: ffmpeg codec (H264 - MPEG-4 AVC (part 10)) started
main debug: using decoder module "avcodec"
main debug: TIMER module_need() : 429.147 ms - Total 429.147 ms / 1 intvls (Avg 429.147 ms)
main debug: looking for packetizer module: 21 candidates
packetizer_h264 debug: found NAL_SPS (sps_id=0)
packetizer_h264 debug: found NAL_PPS (pps_id=0 sps_id=0)
main debug: using packetizer module "packetizer_h264"
main debug: TIMER module_need() : 30.639 ms - Total 30.639 ms / 1 intvls (Avg 30.639 ms)
main debug: thread (decoder) created at priority 0 (input/decoder.c:301)
main debug: thread started
main debug: looking for meta reader module: 2 candidates
lua debug: Trying Lua scripts in 
lua debug: Trying Lua scripts in 
lua debug: Trying Lua playlist script 
lua debug: Trying Lua scripts in 
main debug: no meta reader module matching "any" could be loaded
main debug: TIMER module_need() : 1.195 ms - Total 1.195 ms / 1 intvls (Avg 1.195 ms)
main debug: `<removed>' successfully opened
live555 warning: no data received in 10s. Switching to TCP
avcodec debug: ffmpeg codec (H264 - MPEG-4 AVC (part 10)) stopped
main debug: removing module "avcodec"
main debug: killing decoder fourcc `h264', 0 PES in FIFO
main debug: removing module "packetizer_h264"
main debug: Program doesn't contain anymore ES
live555 debug: RTP subsession 'video/H264'
main debug: looking for decoder module: 31 candidates
avcodec debug: libavcodec already initialized
avcodec debug: trying to use direct rendering
avcodec debug: ffmpeg codec (H264 - MPEG-4 AVC (part 10)) started
main debug: using decoder module "avcodec"
main debug: TIMER module_need() : 4.829 ms - Total 4.829 ms / 1 intvls (Avg 4.829 ms)
main debug: looking for packetizer module: 21 candidates
packetizer_h264 debug: found NAL_SPS (sps_id=0)
packetizer_h264 debug: found NAL_PPS (pps_id=0 sps_id=0)
main debug: using packetizer module "packetizer_h264"
main debug: TIMER module_need() : 0.389 ms - Total 0.389 ms / 1 intvls (Avg 0.389 ms)
main debug: thread (decoder) created at priority 0 (input/decoder.c:301)
main debug: thread started
live555 debug: setup start: 0.000000 stop:0.000000
live555 debug: play start: 0.000000 stop:0.000000
live555 debug: tk->rtpSource->hasBeenSynchronizedUsingRTCP()
main debug: ES_OUT_RESET_PCR called
main debug: Buffering 0%
main debug: Buffering 16%
main debug: Buffering 33%
main debug: no usable vout present, spawning one
main debug: looking for text renderer module: 3 candidates
main debug: using text renderer module "quartztext"
main debug: TIMER module_need() : 20.972 ms - Total 20.972 ms / 1 intvls (Avg 20.972 ms)
main debug: looking for video filter2 module: 15 candidates
swscale debug: 32x32 chroma: YUVA -> 16x16 chroma: YUVA with scaling using Bicubic (good quality)
main debug: using video filter2 module "swscale"
main debug: TIMER module_need() : 24.920 ms - Total 24.920 ms / 1 intvls (Avg 24.920 ms)
main debug: looking for video filter2 module: 15 candidates
yuvp debug: YUVP to YUVA converter
main debug: using video filter2 module "yuvp"
main debug: TIMER module_need() : 44.990 ms - Total 44.990 ms / 1 intvls (Avg 44.990 ms)
main debug: window size: 320x240
main debug: Deinterlacing available
main debug: deinterlace 0, mode blend, is_needed 0
main debug: looking for video output module: 2 candidates
main debug: looking for opengl provider module: 2 candidates
macosx debug: display is Quartz Extreme accelerated
main debug: Buffering 50%
main debug: Buffering 66%
main debug: using opengl provider module "macosx"
main debug: TIMER module_need() : 395.794 ms - Total 395.794 ms / 1 intvls (Avg 395.794 ms)
main debug: using video output module "opengl"
main debug: TIMER module_need() : 407.596 ms - Total 407.596 ms / 1 intvls (Avg 407.596 ms)
main debug: got 1 direct buffer(s)
main debug: pic render sz 320x240, of (0,0), vsz 320x240, 4cc I420, sar 1:1, msk r0x0 g0x0 b0x0
main debug: pic in sz 320x240, of (0,0), vsz 320x240, 4cc I420, sar 1:1, msk r0x0 g0x0 b0x0
main debug: pic out sz 320x240, of (0,0), vsz 320x240, 4cc UYVY, sar 1:1, msk r0x0 g0x0 b0x0
main debug: looking for video filter2 module: 15 candidates
main debug: using video filter2 module "i420_yuy2_sse2"
main debug: TIMER module_need() : 0.287 ms - Total 0.287 ms / 1 intvls (Avg 0.287 ms)
main debug: indirect render, mapping render pictures 0-24 to system pictures 1-25
avcodec debug: using direct rendering
main debug: End of video preroll
main debug: Received first picture
main debug: Post-processing available
main warning: vlc_object_find_name(postproc) is not safe!
main debug: Buffering 83%
main debug: Buffering 100%
main debug: looking for video blending module: 1 candidate
blend debug: chroma: YUVA -> UYVY
main debug: using video blending module "blend"
main debug: TIMER module_need() : 25.443 ms - Total 25.443 ms / 1 intvls (Avg 25.443 ms)
main debug: Stream buffering done (1400 ms in 1232 ms)
main debug: Decoder buffering done in 1 ms
main warning: late picture skipped (108223 > -161)
main debug: incoming request - stopping current input

When it fails I receive this

main debug: `<removed>' successfully opened
live555 warning: no data received in 10s. Switching to TCP
avcodec debug: ffmpeg codec (H264 - MPEG-4 AVC (part 10)) stopped
main debug: removing module "avcodec"
main debug: killing decoder fourcc `h264', 0 PES in FIFO
main debug: removing module "packetizer_h264"
main debug: Program doesn't contain anymore ES
live555 debug: RTP subsession 'video/H264'
main debug: looking for decoder module: 31 candidates
avcodec debug: libavcodec already initialized
avcodec debug: trying to use direct rendering
avcodec debug: ffmpeg codec (H264 - MPEG-4 AVC (part 10)) started
main debug: using decoder module "avcodec"
main debug: TIMER module_need() : 1.817 ms - Total 1.817 ms / 1 intvls (Avg 1.817 ms)
main debug: looking for packetizer module: 21 candidates
packetizer_h264 debug: found NAL_SPS (sps_id=0)
packetizer_h264 debug: found NAL_PPS (pps_id=0 sps_id=0)
main debug: using packetizer module "packetizer_h264"
main debug: TIMER module_need() : 0.288 ms - Total 0.288 ms / 1 intvls (Avg 0.288 ms)
main debug: thread (decoder) created at priority 0 (input/decoder.c:301)
main debug: thread started
live555 debug: setup start: 0.000000 stop:0.000000
live555 debug: play start: 0.000000 stop:0.000000
live555 error: no data received in 10s, aborting
main debug: EOF reached
main debug: finished input
avcodec debug: ffmpeg codec (H264 - MPEG-4 AVC (part 10)) stopped
main debug: removing module "avcodec"
main debug: killing decoder fourcc `h264', 0 PES in FIFO
main debug: removing module "packetizer_h264"
macosx debug: input has stopped, refreshing interface
macosx debug: input has changed, refreshing interface
macosx debug: input has stopped, refreshing interface
main debug: removing module "live555"
main debug: Program doesn't contain anymore ES
main debug: thread ended
main debug: dead input
main debug: TIMER input launching for <removed>' : 1308.189 ms - Total 1308.189 ms / 1 intvls (Avg 1308.189 ms)
main debug: changing item without a request (current 0/1)
main debug: nothing to play

When if fails, which is more often then not, I can pull up the stream from Flash or iOS with no problems. When I am testing I usually have Flash running all the time to make sure the cam is really up.

ffserver VLC output (50kbs stream)

main debug: processing request item <removed> node Playlist skip 0
main debug: resyncing on <removed>
main debug: <removed> is at 0
main debug: starting new item
main debug: creating new input thread
main debug: Creating an input for '<removed>'
main debug: thread (input) created at priority 22 (input/input.c:214)
main debug: thread started
main debug: using timeshift granularity of 50 MiB
main debug: using timeshift path '/tmp'
main debug: `<removed>' gives access `rtsp' demux `' path `<removed>'
main debug: creating demux: access='rtsp' demux='' path='<removed>'
main debug: looking for access_demux module: 1 candidate
macosx debug: input has changed, refreshing interface
live555 debug: RTP subsession 'video/H264'
main debug: selecting program id=0
live555 debug: setup start: 0.000000 stop:0.000000
live555 debug: play start: 0.000000 stop:0.000000
main debug: using access_demux module "live555"
main debug: TIMER module_need() : 643.781 ms - Total 643.781 ms / 1 intvls (Avg 643.781 ms)
main debug: looking for decoder module: 31 candidates
avcodec debug: libavcodec already initialized
avcodec debug: trying to use direct rendering
avcodec debug: ffmpeg codec (H264 - MPEG-4 AVC (part 10)) started
main debug: using decoder module "avcodec"
main debug: TIMER module_need() : 1.628 ms - Total 1.628 ms / 1 intvls (Avg 1.628 ms)
main debug: looking for packetizer module: 21 candidates
packetizer_h264 debug: found NAL_SPS (sps_id=0)
packetizer_h264 debug: found NAL_PPS (pps_id=0 sps_id=0)
main debug: using packetizer module "packetizer_h264"
main debug: TIMER module_need() : 0.289 ms - Total 0.289 ms / 1 intvls (Avg 0.289 ms)
main debug: thread (decoder) created at priority 0 (input/decoder.c:301)
main debug: thread started
main debug: looking for meta reader module: 2 candidates
lua debug: Trying Lua scripts in /Users/fiona/Library/Application Support/org.videolan.vlc/lua/meta/reader
lua debug: Trying Lua scripts in /Applications/VLC.app/Contents/MacOS/share/lua/meta/reader
lua debug: Trying Lua playlist script /Applications/VLC.app/Contents/MacOS/share/lua/meta/reader/filename.lua
lua debug: Trying Lua scripts in /Applications/VLC.app/Contents/MacOS/share/share/lua/meta/reader
main debug: no meta reader module matching "any" could be loaded
main debug: TIMER module_need() : 1.759 ms - Total 1.759 ms / 1 intvls (Avg 1.759 ms)
main debug: `<removed>' successfully opened
main debug: TIMER input launching for '<removed>' : 688.566 ms - Total 688.566 ms / 1 intvls (Avg 688.566 ms)
live555 debug: tk->rtpSource->hasBeenSynchronizedUsingRTCP()
main debug: ES_OUT_RESET_PCR called
main debug: Buffering 0%
main debug: Buffering 41%
main debug: Buffering 41%
main debug: Buffering 41%
main debug: Buffering 41%
main debug: Buffering 83%
main debug: no usable vout present, spawning one
main debug: looking for text renderer module: 3 candidates
main debug: using text renderer module "quartztext"
main debug: TIMER module_need() : 0.361 ms - Total 0.361 ms / 1 intvls (Avg 0.361 ms)
main debug: looking for video filter2 module: 15 candidates
swscale debug: 32x32 chroma: YUVA -> 16x16 chroma: YUVA with scaling using Bicubic (good quality)
main debug: using video filter2 module "swscale"
main debug: TIMER module_need() : 0.932 ms - Total 0.932 ms / 1 intvls (Avg 0.932 ms)
main debug: looking for video filter2 module: 15 candidates
yuvp debug: YUVP to YUVA converter
main debug: using video filter2 module "yuvp"
main debug: TIMER module_need() : 0.285 ms - Total 0.285 ms / 1 intvls (Avg 0.285 ms)
main debug: window size: 320x240
main debug: Deinterlacing available
main debug: deinterlace 0, mode blend, is_needed 0
main debug: looking for video output module: 2 candidates
main debug: looking for opengl provider module: 2 candidates
macosx debug: display is Quartz Extreme accelerated
main debug: using opengl provider module "macosx"
main debug: TIMER module_need() : 25.530 ms - Total 25.530 ms / 1 intvls (Avg 25.530 ms)
main debug: using video output module "opengl"
main debug: TIMER module_need() : 25.856 ms - Total 25.856 ms / 1 intvls (Avg 25.856 ms)
main debug: got 1 direct buffer(s)
main debug: pic render sz 320x240, of (0,0), vsz 320x240, 4cc I420, sar 1:1, msk r0x0 g0x0 b0x0
main debug: pic in sz 320x240, of (0,0), vsz 320x240, 4cc I420, sar 1:1, msk r0x0 g0x0 b0x0
main debug: pic out sz 320x240, of (0,0), vsz 320x240, 4cc UYVY, sar 1:1, msk r0x0 g0x0 b0x0
main debug: looking for video filter2 module: 15 candidates
main debug: using video filter2 module "i420_yuy2_sse2"
main debug: TIMER module_need() : 0.376 ms - Total 0.376 ms / 1 intvls (Avg 0.376 ms)
main debug: indirect render, mapping render pictures 0-24 to system pictures 1-25
avcodec debug: using direct rendering
main debug: End of video preroll
main debug: Received first picture
main debug: looking for video blending module: 1 candidate
blend debug: chroma: YUVA -> UYVY
main debug: using video blending module "blend"
main debug: TIMER module_need() : 0.453 ms - Total 0.453 ms / 1 intvls (Avg 0.453 ms)
main debug: Post-processing available
main warning: vlc_object_find_name(postproc) is not safe!
main debug: Stream buffering done (1500 ms in 1401 ms)
main debug: Decoder buffering done in 0 ms
main warning: late picture skipped (300912 > -559)
main warning: late picture skipped (146741 > -192)
2

My understanding is that the Flash player needs 64 frames to start playback. So the higher frame rate is going to help. Another option is to set the client side buffer (NetSTream.setBufferTime(seconds)) to zero it might help things startup more quickly and I think it will play each frame as it comes in.

Charlie

3

Did you try what I suggested? Setting the NetStream.bufferTime in your Flash player to zero?

Charlie

4

I think it is more a player issue than a Wowza issue. With the low framerate the players are timing out. I don’t think there is anything you can change in Wowza.

One other consideration is key frame interval. Wowza will only start sending video on the first key frame. So with a low frame rate you need to be sure you are sending key frame every 1-3 seconds.

Charlie

5

Cool, glad to hear you got this sorted out.

Charlie

6

I did a test with an Axis cam by setting it to 1FPS with a constant bit rate of 50kbs. The same thing occurs. I can stream RTSP directly from the camera, but not through Wowza.

I imagine what I am trying to do is really outside the scope of what Wowza is intended to do. We are just trying to keep our bandwidth costs to a minimum. Up until this point our users have streamed directly from our cams which are on fixed cost residential accounts (florida surfcams).

7

I have not tried this out yet. I am still working on getting the ffserver/ffmpeg RTSP/RTP stream to work through Wowza. It works fine when streaming from ffserver. Once I stream from ffserver through Wowza it no longer works (it will work very rarely). HTTP iOS and RTMP works fine. I can get it to work but it is a hack. I set the frame rate to 15 in ffserver.conf. It appears as though ffmpeg will then force this frame rate and reuse existing images. Once I fix the frame rate at what it is (around 4), RTSP/RTP streaming no longer works through Wowza (again it will work very rarely). If I watch my network when starting a stream using the flash control I notice for around 5 to 10 seconds there is less then 1KBs input. Then it jumps as the video data is delivered. VLC always times out.

At this point I am not sure if it is a Wowza configuration issue, or the H.264 transcoded stream. I understand I am taking a raw image stream and encoding it into a time based format. We are just trying to avoid having to replace 15 canon cameras with Axis cameras. A rough estimate puts it close to 25k. Not something we want to do over night. We already have a few Axis cameras and that is what lead me here. I happened to get ffserver to transcode and stream to VLC and Android. I had hoped it would work through Wowza.

I am going to leave my servers running with a configured stream that illustrates the issue. I have sent the stream info in a PM if you want to take a look.

8

In WireShark

For Wowza -

I receive 3 ICMP messages of Type 3 Code 3 (Destination and Port Unreachable).

The stream is setup and torn down twice

For ffserver -

I recieve 1 ICMP message of Type 3 COde 3.

THe stream is setup and torn down once. After this is streams.

The first connection attempt fails. After this VLC tries TCP. This works with FFServer but does not with Wowza. I’ll compare SDP files and take a look at the handshaking a little closer.

9

There is a strange issue with RTSP/RTP and Wowza. I am encoding at 3fps which comes out to about a 20kps stream. If I stream this through Wowza it takes a considerable amount of time for the RTMP stream to come through on flash. I mean data wise. Now if I instruct ffserver to stream the file at 15 fps which comes out to about 200kbps stream, Wowza becomes happy and everything works great. Even then though RTSP takes around 35 seconds of connecting and buffering on Android (20 seconds on VLC). This is due to RTP UDP failing and falling back to TCP.

Is there a way to just configure Wowza as a pass through for RTSP? Any ideas as to where to look to find out what is going on?

10

Thank you. Setting -g 3 in my ffmpeg command was all that was needed. For some reason I still have to set VideoFrameRate to 15 in my ffserver.conf. But setting VideoBitRate at 20 works where as it did not before.

Thanks again for the help.

11

Well this somewhat sorted the issue. I found that having 12 ffmpeg instances running on a small EC2 instance was another issue all together. Moving to a Compute medium made the streams a bit more reliable. The ffmpeg -g option is not sending out I frames as I had thought. I still had to set ffserver to 15fps for the stream configuration. What I have done to solve the issue once and for all is to modify ffmpeg. I downloaded the latest svn release. This version has the force_key_frames option. However, this option forces I frames at specific times. I simply modified the code to send out an I frame every 3 frames. So far this works alright. VLC does complain a bit about late frames. RTMP streaming is immediate. It used to take around 9 seconds for the stream to come in. Now it takes 1.