Injecting "closely synced" live captions via AMF onTextData: Timecode alignment?

Wowza Developer Dojo
1

I’m building a Proof of Concept (Poc) for a client that uses a Flash SWF front-end to send live captions to a live stream being ingested by Wowza. I’m no stranger to Wowza module development, but admittedly, I haven’t dived into timecode data with server-side streams. I’ve read more than a couple of threads on the forum already related to cue point injection (same issue really), but I’m still left wondering how I can take my PoC further. Specifically, I would like to adjust for the latency in the captioner’s front-end when sending the “on the fly” captions. I understand that the captions will always be behind / delayed by nature of a live broadcast being transcribed, but I’d like to see if I can have them “less delayed” by injecting AMF onTextData at an “earlier” buffered state of the live stream on the Wowza stream. This presumes, of course, that I can set an internal buffer (say, up to 10 seconds if necessary) on which onTextData information can be applied, before being sent out by the server. I’ve considered the possibility too that nDVR creates a more “natural” buffer for a live stream, just by nature of playlists and chunks being created during the streaming process. So, I’m looking for any advice to take any offset information I can gather in the captioner (timecode offset between the publisher and the consuming swf can be accounted for, as well as latency from the SWF to the Wowza server). Here’s my current PoC:

http://encoding.videorx.com/clients/bcv/captioninput.html

Also, my “receiver” method in the Wowza module I’ve written is fairly straightforward:

public void onCaptionText(IClient client, RequestFunction function,

AMFDataList params) {

getLogger().info(“CaptionInputReceiver:nCaptionText() invoked”);

AMFDataObj param1 = (AMFDataObj)getParam(params, PARAM1);

String text = param1.getString(“text”);

String streamName = param1.getString(“stream”);

String lang = param1.getString(“lang”);

int trackId = param1.getInt(“trackid”);

MediaStreamMap streams = appInstance.getStreams();

IMediaStream stream = streams.getStream(streamName);

AMFDataObj amfData = new AMFDataObj();

amfData.put(“text”, new AMFDataItem(text));

amfData.put(“language”, new AMFDataItem(lang));

amfData.put(“trackid”, new AMFDataItem(trackId));

stream.sendDirect(“onTextData”, amfData);

}

Many thanks in advance for any insights!

2

As a follow-up to my original post, I’ve since changed my approach to use the ModulePublishWithDelay module to delay a published stream, and attempted to inject AMF data into the “proxy” delayed stream created by the module. The approach works, but I’m trying to compensate for the latency and buffer of the original stream as watched by the person typing the captions. Is there a way to get the AMFPacket.getAbsTimecode() value in client-side ActionScript on the currently displayed frame during NetStream playback?

3

I think you would have to inject them as cuepoints into the stream.

Richard