Live streaming AV drift from Mediacaster/UDP source

Hi guys,

I am running into a weird problem and can’t seems find a way to fix it, any help is much appreciated.

Source, Wowza, re-streaming systrem, and playback devices are all on local network. Under the same environment, I grab HTTP source video stream using either VLC or FFMPEG, but when re-stream into Wowza using Mediacaster/MPEG-TS/UDP method, I am getting AV drift after just few minutes of playback. No transcoding is done on FFMPEG or VLC.

This problem doesn’t happen if I re-stream into Wowza using RTMP with FFMPEG.

Source is H264 with MP3. Playback is on few different Alibaba-sourced Android media box, using built-in video player, with Android 4.2.2. Streaming method is with HLS. For RTP/AVSyncMethod, I had tried all 3 with same result.

Also tried adding av sync related properties into application.xml but still the same,

https://www.wowza.com/docs/how-to-fix-unaligned-video-and-audio-with-a-server-side-sort-buffer

https://www.wowza.com/docs/how-to-publish-and-play-a-live-stream-mpeg-ts-based-encoder

Complete application.xml as follow,

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<Root>
	<Application>
		<!-- Uncomment to set application level timeout values
		<ApplicationTimeout>60000</ApplicationTimeout>
		<PingTimeout>12000</PingTimeout>
		<ValidationFrequency>8000</ValidationFrequency>
		<MaximumPendingWriteBytes>0</MaximumPendingWriteBytes>
		<MaximumSetBufferTime>60000</MaximumSetBufferTime>
		<MaximumStorageDirDepth>25</MaximumStorageDirDepth>
		-->
		<Connections>
			<AutoAccept>true</AutoAccept>
			<AllowDomains/>
		</Connections>
		<!--
			StorageDir path variables
			
			${com.wowza.wms.AppHome} - Application home directory
			${com.wowza.wms.ConfigHome} - Configuration home directory
			${com.wowza.wms.context.VHost} - Virtual host name
			${com.wowza.wms.context.VHostConfigHome} - Virtual host config directory
			${com.wowza.wms.context.Application} - Application name
			${com.wowza.wms.context.ApplicationInstance} - Application instance name
			
		-->
		<Streams>
			<StreamType>live</StreamType>
			<StorageDir>${com.wowza.wms.context.VHostConfigHome}/content</StorageDir>
			<KeyDir>${com.wowza.wms.context.VHostConfigHome}/keys</KeyDir>
			<!-- LiveStreamPacketizers (separate with commas): cupertinostreamingpacketizer, smoothstreamingpacketizer, sanjosestreamingpacketizer, cupertinostreamingrepeater, smoothstreamingrepeater, sanjosestreamingrepeater, dvrstreamingpacketizer, dvrstreamingrepeater -->
			<LiveStreamPacketizers>cupertinostreamingpacketizer</LiveStreamPacketizers>
			<!-- Properties defined here will override any properties defined in conf/Streams.xml for any streams types loaded by this application -->
			<Properties>
<!-- https://www.wowza.com/docs/how-to-fix-unaligned-video-and-audio-with-a-server-side-sort-buffer -->
<!-- How to fix unaligned video and audio with a server-side sort buffer  -->
<Property>
    <Name>sortPackets</Name>
    <Value>true</Value>
    <Type>Boolean</Type>
</Property>
<Property>
    <Name>sortBufferSize</Name>
    <Value>7500</Value>
    <Type>Integer</Type>
</Property>
			</Properties>
		</Streams>
		<Transcoder>
			<!-- To turn on transcoder set to: transcoder -->
			<LiveStreamTranscoder/>
			<!-- [templatename].xml or ${SourceStreamName}.xml -->
			<Templates>${SourceStreamName}.xml,transrate.xml</Templates>
			<ProfileDir>${com.wowza.wms.context.VHostConfigHome}/transcoder/profiles</ProfileDir>
			<TemplateDir>${com.wowza.wms.context.VHostConfigHome}/transcoder/templates</TemplateDir>
			<Properties>
			</Properties>
		</Transcoder>
		<DVR>
			<!-- As a single server or as an origin, use dvrstreamingpacketizer in LiveStreamPacketizers above -->
			<!-- Or, in an origin-edge configuration, edges use dvrstreamingrepeater in LiveStreamPacketizers above -->
			<!-- As an origin, also add dvrchunkstreaming to HTTPStreamers below -->
			<!-- To turn on DVR recording set Recorders to dvrrecorder.  This works with dvrstreamingpacketizer  -->
			<Recorders/>
			<!-- As a single server or as an origin, set the Store to dvrfilestorage-->
			<!-- edges should have this empty -->
			<Store/>
			<!--  Window Duration is length of live DVR window in seconds.  0 means the window is never trimmed. -->
			<WindowDuration>0</WindowDuration>
			<!-- Storage Directory is top level location where dvr is stored.  e.g. c:/temp/dvr -->
			<StorageDir>${com.wowza.wms.context.VHostConfigHome}/dvr</StorageDir>
			<!-- valid ArchiveStrategy values are append, version, delete -->
			<ArchiveStrategy>append</ArchiveStrategy>
			<!-- If this is a dvrstreamingrepeater, -->
			<!-- either define the general Application/Repeater/OriginURL below to point back to the origin -->
			<!-- or define this one (which affects only DVR origin-edge).  e.g. http://localhost/dvrorigin -->
			<Repeater>
				<ChunkOriginURL/>
			</Repeater>
			<!-- Properties for DVR -->
			<Properties>
			</Properties>
		</DVR>
		<TimedText>
			<!-- VOD caption providers (separate with commas): vodcaptionprovidermp4_3gpp, vodcaptionproviderttml, vodcaptionprovidersrt, vodcaptionproviderscc -->
			<VODTimedTextProviders>vodcaptionprovidermp4_3gpp</VODTimedTextProviders>
			
			<!-- Properties for TimedText -->
			<Properties>
			</Properties>		
		</TimedText>
		
		<!-- HTTPStreamers (separate with commas): cupertinostreaming, smoothstreaming, sanjosestreaming, dvrchunkstreaming -->
		<HTTPStreamers>cupertinostreaming</HTTPStreamers>
		<SharedObjects>
			<StorageDir/>
		</SharedObjects>
		<Client>
			<IdleFrequency>-1</IdleFrequency>
			<Access>
				<StreamReadAccess>*</StreamReadAccess>
				<StreamWriteAccess>*</StreamWriteAccess>
				<StreamAudioSampleAccess/>
				<StreamVideoSampleAccess/>
				<SharedObjectReadAccess>*</SharedObjectReadAccess>
				<SharedObjectWriteAccess>*</SharedObjectWriteAccess>
			</Access>
		</Client>
		<RTP>
			<!-- RTP/Authentication/[type]Methods defined in Authentication.xml. Default setup includes; none, basic, digest -->
			<Authentication>
				<PublishMethod>digest</PublishMethod>
				<PlayMethod>digest</PlayMethod>
			</Authentication>
			<!-- RTP/AVSyncMethod. Valid values are: senderreport, systemclock, rtptimecode -->
			<AVSyncMethod>rtptimecode</AVSyncMethod>
			<MaxRTCPWaitTime>12000</MaxRTCPWaitTime>
			<IdleFrequency>75</IdleFrequency>
			<RTSPSessionTimeout>90000</RTSPSessionTimeout>
			<RTSPMaximumPendingWriteBytes>0</RTSPMaximumPendingWriteBytes>
			<RTSPBindIpAddress/>
			<RTSPConnectionIpAddress>0.0.0.0</RTSPConnectionIpAddress>
			<RTSPOriginIpAddress>127.0.0.1</RTSPOriginIpAddress>
			<IncomingDatagramPortRanges>*</IncomingDatagramPortRanges>
			<!-- Properties defined here will override any properties defined in conf/RTP.xml for any depacketizers loaded by this application -->
			<Properties>
<!-- https://www.wowza.com/docs/how-to-publish-and-play-a-live-stream-mpeg-ts-based-encoder(MPEG-TS-based-encoder) -->
<!-- Properties to fix audio and video alignment -->
<Property>
    <Name>audioIsAligned</Name>
    <Value>false</Value>
    <Type>Boolean</Type>
</Property>
<Property>
    <Name>videoIsAligned</Name>
    <Value>false</Value>
    <Type>Boolean</Type>
</Property>
<!-- https://www.wowza.com/docs/how-to-publish-and-play-a-live-stream-mpeg-ts-based-encoder(MPEG-TS-based-encoder) -->
<!-- Property to fix TS packet alignment in UDP packets -->
<Property>
    <Name>rtpDePacketizerWrapper</Name>
    <Value>com.wowza.wms.rtp.depacketizer.RTPDePacketizerWrapperReChunkMPEGTS</Value>
</Property>
<!-- http://community.wowza.com/t/-/33683 -->
<!-- For RTP/AVSyncMethod=senderreport, sync on subsequent SR packet, instead of just initial SR packet  -->
<Property>
    <Name>rtpResyncAudioVideoOnSR</Name>
    <Value>true</Value>
    <Type>Boolean</Type>
</Property>
			</Properties>
		</RTP>
		<MediaCaster>
			<RTP>
				<RTSP>
					<!-- udp, interleave -->
					<RTPTransportMode>interleave</RTPTransportMode>
				</RTSP>
			</RTP>
			<!-- Properties defined here will override any properties defined in conf/MediaCasters.xml for any MediaCasters loaded by this applications -->
			<Properties>
			</Properties>
		</MediaCaster>
		<MediaReader>
			<!-- Properties defined here will override any properties defined in conf/MediaReaders.xml for any MediaReaders loaded by this applications -->
			<Properties>
			</Properties>
		</MediaReader>
		<MediaWriter>
			<!-- Properties defined here will override any properties defined in conf/MediaWriter.xml for any MediaWriter loaded by this applications -->
			<Properties>
			</Properties>
		</MediaWriter>
		<LiveStreamPacketizer>
			<!-- Properties defined here will override any properties defined in conf/LiveStreamPacketizers.xml for any LiveStreamPacketizers loaded by this applications -->
			<Properties>
<!-- https://www.wowza.com/docs/how-to-turn-off-data-event-processing-for-apple-hls-streams -->
<Property>
	<Name>cupertinoEnableDataEvents</Name>
	<Value>false</Value>
	<Type>Boolean</Type>
</Property>
<!-- https://www.wowza.com/docs/how-to-configure-apple-hls-packetization-cupertinostreaming(cupertinostreaming) -->
<Property>
	<Name>cupertinoChunkDurationTarget</Name>
	<Value>10000</Value>
	<Type>Integer</Type>
</Property>
<Property>
	<Name>cupertinoMaxChunkCount</Name>
	<Value>10</Value>
	<Type>Integer</Type>
</Property>
<Property>
	<Name>cupertinoPlaylistChunkCount</Name>
	<Value>3</Value>
	<Type>Integer</Type>
</Property>
<Property>
	<Name>cupertinoRepeaterChunkCount</Name>
	<Value>3</Value>
	<Type>Integer</Type>
</Property>
			</Properties>
		</LiveStreamPacketizer>
		<HTTPStreamer>
			<!-- Properties defined here will override any properties defined in conf/HTTPStreamers.xml for any HTTPStreamer loaded by this applications -->
			<Properties>
			</Properties>
		</HTTPStreamer>
		<Repeater>
			<OriginURL/>
			<QueryString/>
		</Repeater> 
		<Modules>
			<Module>
				<Name>base</Name>
				<Description>Base</Description>
				<Class>com.wowza.wms.module.ModuleCore</Class>
			</Module>
			<Module>
				<Name>properties</Name>
				<Description>Properties</Description>
				<Class>com.wowza.wms.module.ModuleProperties</Class>
			</Module>
			<Module>
				<Name>logging</Name>
				<Description>Client Logging</Description>
				<Class>com.wowza.wms.module.ModuleClientLogging</Class>
			</Module>
			<Module>
				<Name>flvplayback</Name>
				<Description>FLVPlayback</Description>
				<Class>com.wowza.wms.module.ModuleFLVPlayback</Class>
			</Module> 
		
			<Module><Name>Wmsauth</Name><Description>Wmsauth</Description><Class>com.wmspanel.Wmsauth</Class></Module>
		<Module><Name>ModuleRTMPAuthenticate</Name><Description>ModuleRTMPAuthenticate</Description><Class>com.wowza.wms.security.ModuleRTMPAuthenticate</Class></Module></Modules>
		<!-- Properties defined here will be added to the IApplication.getProperties() and IApplicationInstance.getProperties() collections -->
		<Properties>
		</Properties>
	</Application>
</Root>

Hi there, you may be able to remedy this by adding a jitter buffer, and you can log packet loss if it is occurring by following this guide:

How to turn on an RTP jitter buffer and packet loss logging (RTP and MPEG-TS)

If you find you are having packet loss you can see if making these server adjustments corrects the issue:

How to fix UDP packet loss (MPEG-TS/RTP)

Salvadore

Since it’s a live system, I haven’t being able to put the required changes. But given it was tested all on local network connecting to same network switch, I am not sure there would be any kind of packet lost?

Further to the test that was made in first page, I find that if I playback in VLC, even the StreamManager/MPEG-TS/UDP method would not have any audio/video sync problem. The funny part is that I see the monitor with Android STB, the audio played is matching exactly the same as VLC, but video lacks several seconds behind. This would go on for several minutes then suddenly a short stop and comes back correctly, just to slowly drift apart again.

To me it would appears it’s problem with STB but somehow RTMP method works correctly… So it comes down to what’s missing in StreamManager/MPEG-TS/UDP method that is somehow availalbe in RTMP method?

Salvadore, thanks for your help. Being trying various settings you suggested and still having same issue…

I even tried this and still RTMP stream is OK…

http source -> vlc or ffmpeg re-stream by rtp/ts -> Wowza mediacaster -> Wowza HLS stream -> ffmpeg re-stream by rtmp -> same Wowza…

As long as the stream get touched by RTMP, it’s magically all OK…

I also tried various RTP timecode settings and doesn’t seems to make any difference…

What else could be the issue? Any suggestion and help is much appreciated…