Using gstreamer-0.10 with Wowza

Wowza Streaming Engine
1

Hi

I’ve begun experimenting using Wowza together with the gstreamer media framework.

Here’s the cmd line i’m using:

gst-launch-0.10 -vvv videotestsrc ! queue ! x264enc byte-stream=true bitrate=300 ! rtph264pay ! udpsink port=5000 host=127.0.0.1 sync=false

It results in creating a 320x240 video test pattern encoded using x264.

Since there’s no RTCP enabled with such command line, i put the systemclock sync parameter in the Application.xml config file.

Server-side (on the local host), here’s the myStream.sdp file:

v=0

o=- 1188340656180883 1 IN IP4 127.0.0.1

s=Session streamed by GStreamer

i=server.sh

t=0 0

a=tool:GStreamer

a=type:broadcast

m=video 5000 RTP/AVP 96

c=IN IP4 127.0.0.1

a=rtpmap:96 H264/90000

Here’s the debug log (relevant parts):

DEBUG server comment - cmd: play

INFO server comment - MediaStreamMediaCasterPlay: startPlay

INFO server comment - RTPMediaCaster.create

INFO server comment - RTPMediaCaster.init

DEBUG server comment - cmd: setBufferTime

INFO server comment - RTPMediaCaster.Reconnector: start

INFO server comment - RTPSessionDescriptionDataProviderBasic.getStreamInfo: /usr/local/WowzaMediaServerPro/content/myStream.sdp

DEBUG server comment - sdp: v=0

DEBUG server comment - sdp: o=- 1188340656180883 1 IN IP4 127.0.0.1

DEBUG server comment - sdp: s=Session streamed by GStreamer

DEBUG server comment - sdp: i=server.sh

DEBUG server comment - sdp: t=0 0

DEBUG server comment - sdp: a=tool:GStreamer

DEBUG server comment - sdp: a=type:broadcast

DEBUG server comment - sdp: m=video 5000 RTP/AVP 96

DEBUG stream setbuffertime myStream.sdp -

DEBUG server comment - sdp: c=IN IP4 127.0.0.1

DEBUG server comment - sdp: a=rtpmap:96 H264/90000

DEBUG server comment - onFlushNotifyClients: false

DEBUG server comment - flushInterval: 0

DEBUG server comment - verboseDebug: false

INFO stream create - -

DEBUG server comment - RTPDePacketizerRFC3984H264.init

INFO stream publish myStream.sdp -

INFO server comment - UDPTransport.bind: /127.0.0.1:5000

DEBUG server comment - config: session: setReuseAddress: from:false to:true

DEBUG server comment - config: session: setReceiveBufferSize: from:55296 to:65000

DEBUG server comment - config: session: setSendBufferSize: from:55296 to:55296

DEBUG server comment - config: session: setTrafficClass: from:0 to:0

INFO server comment - UDPTransport.bind: /127.0.0.1:5001

DEBUG server comment - config: session: setReuseAddress: from:false to:true

DEBUG server comment - config: session: setReceiveBufferSize: from:55296 to:65000

DEBUG server comment - config: session: setSendBufferSize: from:55296 to:55296

DEBUG server comment - config: session: setTrafficClass: from:0 to:0

INFO server comment - RTPMediaCaster.Reconnector: stop

DEBUG server comment - rtp[video:1024] {80 60 11 af 25 e9 6f d6 e3 2f 49 73 1c 89 30 00 }

DEBUG server comment - myStream.sdp: 17:36:15: waitforend: dropped:1

INFO server comment - UDPTransport.firstPacket: /127.0.0.1:5000

DEBUG server comment - rtp[video:286] {80 60 11 b0 25 e9 6f d6 e3 2f 49 73 1c 49 c7 bf }

DEBUG server comment - myStream.sdp: 17:36:15: waitforend: dropped:2

DEBUG server comment - rtp[video:1024] {80 60 11 b1 25 e9 7b 8f e3 2f 49 73 1c 89 10 00 }

DEBUG server comment - myStream.sdp: 17:36:15: waitforend: dropped:3

DEBUG server comment - rtp[video:990] {80 60 11 b2 25 e9 7b 8f e3 2f 49 73 1c 49 de 0a }

DEBUG server comment - myStream.sdp: 17:36:15: waitforend: dropped:4

DEBUG server comment - rtp[video:1024] {80 60 11 b3 25 e9 87 46 e3 2f 49 73 1c 89 30 00 }

DEBUG server comment - myStream.sdp: 17:36:15: waitforend: dropped:5

DEBUG server comment - rtp[video:148] {80 60 11 b4 25 e9 87 46 e3 2f 49 73 1c 49 49 9f }

DEBUG server comment - myStream.sdp: 17:36:15: waitforend: dropped:6

DEBUG server comment - rtp[video:1024] {80 60 11 b5 25 e9 92 fe e3 2f 49 73 1c 89 30 00 }

DEBUG server comment - myStream.sdp: 17:36:15: waitforend: dropped:7

DEBUG server comment - rtp[video:177] {80 60 11 b6 25 e9 92 fe e3 2f 49 73 1c 49 7a 2f }

Of course this won’t be an easy task, yet i’m wondering what this log mean ? Why are all frames dropped ?

Cheers

Florent

2

Also, describe how to install gstreamer. I can’t get it to work with x264enc. I am on Fedora6.

Charlie

3

I really don’t have any new information here. The format of the sprop-parameter-sets looks OK to me. I think there is an issue with Wowza Pro not being able to recognize keys frames in the stream coming from gstreamer.

Charlie

4

I have sent you an email directly. Wowza Pro is not receiving UDP packets from the encoder. So you need to sort this out first.

Charlie

5

I don’t think you are transcoding the content to H.264 in your gstreamer command. It needs to be somethine like:

gst-launch-0.10.exe -v filesrc location=videotestsrc_h264.avi ! avidemux ! x264enc byte-stream=true bitrate=300 ! rtph264pay mtu=1438 ! udpsink host=10.218.35.135 port=1234

Charlie

6

I have been looking at this and I am pretty sure the bug is with gstreamer. There is code in the rtph264pay system that is searching for H.264 start codes. It is looking for start codes in the format:

0x00000001

Start codes in H.264 are only three bytes long. Like this:

0x000001

So it is not properly chopping up the NAL units to send them over RTP. Some RTP handlers can handle this. We cannot. I am hesistant to add support for this since it does not seem to be the standard way to do it. I was planning on attempting to fix the problem in gstreamer to validate my findings but have not had time to get a working gstreamer development setup.

Charlie

7

Thanks for the development instructions. I will try to get back to this to figure out the problem.

I did make a change that fixes the test pattern but it does not seem to universally fix the issue. I have since found other gstreamer setups that do not work even with this fix in place. Again, it has to do with the way gstreamer is breaking apart (or not breaking part) the H.264 packets. I can seem that VLC is able to stream data in this format but it just doesn’t seem correct to me.

Charlie

8

So I am confused. Is it all working now with the most recent version of gstreamer?

Charlie

9

Upgrade the most recent patch:

https://www.wowza.com/devbuild.html

I believe it only works with the most recent version of GStreamer.

Charlie

10

How long does it take for it to go out of sync? RTCP is used to synchronize the video and audio but is only needed to initially sync the stream. MPEG-TS synchronizes the audio and video in the stream (all streams run on a 90KHz clock and are pre-synchronized). Not sure why your stream is going out of sync.

Charlie

11
  1. why do i need to restart the server after stopping and restarting the (publisher) streaming process ? Is there a workaround ? If i do not, the flash player only shows a black screen

It is because when the encoder starts/stop it changes the time basis which screws up Wowza Pro and Flash. There are two methods to work around this. Take a look at the sdpFileCheckFreqency and streamTimeout properties as discussed in the Native Real-time Transport Protocol (RTP) Streaming section of the User’s Guide. You can setup Wowza Pro to monitor the SDP file for date or size changes (sdpFileCheckFreqency) so the stream is restarted when the SDP file changes. You can instruct Wowza Pro monitor the stream for long breaks (streamTimeout) and restart the stream if needed. The streamTimeout is tricky to get right since if it is too short it will restart the stream when it first tries to connect if it can’t find a key frame quickly enough.

  1. audio plays back faster

  2. my audio tests samples (voice/claps) seem to have a direct effect on drifting speed ! If i speak longer, the desync comes faster !

This is a tough one. I just don’t have the resources to dig into gstreamer and debug something like this. I know there are plenty of RTP based encoders where there is no issue with AV sync. So it is most likely not a Wowza Pro issue. There may be rounding issues in GStreamer. I have seen this problem in other encoders where they don’t deal with timecodes properly and they calculate time differentials which insert small rounding errors that add up over time.

Charlie

12

Great!

Charlie

13

These type of issues are usually due to av sync issues. Take a look at this post to see if any of the suggestions here help:

http://community.wowza.com/t/-/57

Charlie

14

I have personally not had much luck with gstreamer. Other then what I have suggested I do not know any magic bullets.

Charlie

15

I changed the resolution to 640x480:

gst-launch-0.10 -vvv videotestsrc ! capsfilter caps=“video/x-raw-yuv, format=(fourcc)I420, width=(int)640, height=(int)480, framerate=(fraction)30/1” ! queue ! x264enc byte-stream=true bitrate=300 ! rtph264pay ! udpsink port=5000 host=127.0.0.1 sync=false

Now i’m getting these messages (again, using systemclock sync):

DEBUG server comment - FU-A: end:false tc:3371173062 st:true

DEBUG server comment - rtp[video:212] {80 e0 05 8f c8 f0 04 c6 3f e9 f1 e0 5c 41 ee c5 }

DEBUG server comment - FU-A: end:true tc:3371173062 st:false

DEBUG server comment - rtp[video:14] {80 60 05 90 c8 f0 10 7f 3f e9 f1 e0 09 30 }

DEBUG server comment - SINGLE: end:false tc:3371176063 sz:2

DEBUG server comment - rtp[video:1400] {80 60 05 91 c8 f0 10 7f 3f e9 f1 e0 5c 81 9a d9 }

DEBUG server comment - FU-A: end:false tc:3371176063 st:true

DEBUG server comment - rtp[video:512] {80 e0 05 92 c8 f0 10 7f 3f e9 f1 e0 5c 41 7a e5 }

DEBUG server comment - FU-A: end:true tc:3371176063 st:false

DEBUG server comment - rtp[video:14] {80 60 05 93 c8 f0 1c 36 3f e9 f1 e0 09 30 }

DEBUG server comment - SINGLE: end:false tc:3371179062 sz:2

DEBUG server comment - rtp[video:1400] {80 60 05 94 c8 f0 1c 36 3f e9 f1 e0 5c 81 9a dd }

My questions are the following:

  • can i assume that wowza is getting video frames correctly ?
  • is it possible to get detailed comments/explanation about the debug messages ?
16

The SDP data does not contain the sprop-parameter-sets parameter. This line usually looks like this in the SDP data.

a=fmtp:96 packetisation-mode=0; profile-level-id=42801e; sprop-parameter-sets=Z01AHuigsPbB6kBAQGQkAAADAAQABX5BgAAHoQAAFuNm974PihM0,aP91IA==

Without this information Wowza Pro must hunt in the NAL units of the H.264 packets for this data. It can’t start video playback until it finds this information. This information is not always contained within the stream.

I really don’t have a specific answer for you. I just know that this is what it seems to be searching for.

Charlie

17

First, try it with Wowza Pro 1.5.1. You can download the upgrade patch here:

https://www.wowza.com/devbuild.html

If that doesn’t work, follow these instructions and send me the debug log files and the command line you are using:

Note: If you experience problems getting either the audio or video to play through Flash, double check the version number of the Flash player (Flash player version 9.0.115.0 or above is required). If you still have problems, turn on Wowza Pro debug logging (edit [install-dir]/conf/log4j.properties and change the log4j.rootCategory on the first line from INFO to DEBUG), try the encoder several more times, zip up and send your [install-dir]/logs folder along with screen shots of the encoder setup screens and the LiveVideoStreaming player screen and send a detailed description of your problem to support@wowza.com.

Charlie

18

Here is SDP data from Wirecast:

v=0
o=- 1798860268 1798860268 IN IP4 127.0.0.0
s=Wirecast
t=0 0
a=range:npt=now-
m=audio 5432 RTP/AVP 96
c=IN IP4 127.0.0.1
a=rtpmap:96 mpeg4-generic/32000/1
a=fmtp:96 profile-level-id=15;mode=AAC-hbr;sizelength=13;indexlength=3;indexdeltalength=3;config=1288
m=video 5434 RTP/AVP 97
c=IN IP4 127.0.0.1
a=rtpmap:97 H264/90000
a=fmtp:97 packetization-mode=1;profile-level-id=4D400C;sprop-parameter-sets=J01ADKkYKD9gDUGAQa2wrXvfAQ==,KN4JF6A=
a=cliprect:0,0,240,320
a=framesize:97 320-240
b=AS:200

It looks like the backslashes could be a problem. If you want me to look at debugs to see if it is a key frame problem, I can.

Charlie

19

Send me the entire debug log. Send it to charlie@wowza.com.

Charlie

20

Be sure you are properly setting the “M” bit in the RTP header of the video packets. We require this so we can identify the starts of packets.

Charlie