I’ve been researching ways how to ingest AAC / opus packets through the packet listener and send them for ingest to another service.
The opus transcoder is containlerless, and needs to be wrapped in an ogg container to be read properly it took ages to figure out why that was failing. So after that failure ive tried passing the AAC packets from the RTMP encoder through ffmpeg to create the ogg/opus. And the incoming AAC packets are corrupted when just recording them. If I pull an rtmp stream into ffmpeg to grab frames it works. The Opus output was corrupting also and wont even play back in VLC.
There is hidden undocumented apis in wowza to work with opus and AAC that might be useful but are hidden and not useful for developers.
I now know there is 2 byte headers in the Opus packet that needs to be stripped but even doing so the service fails to read them because its containless. And trying to wrap them with a Java ogg/opus library failed.
What special treatment do I need to do with the AAC packets to get the raw samples into a ByteBuffer for processing and ingest into ffmpeg to be readable by the service ? The output required is a 16khz 24kps mono output.
The code looks like this. I need a way to detect the input samplerate and channels also. The output of this is noisy and corrupted but at least it plays.
We thought just getting the output bytes direct from the transcoder would work but they are unusable without further processing. So may aswell pass them to ffmpeg to make them usable. But doing so the raw bytes from the audio packet is still unusable without further treatment also. And there is no documentation what it needs.
public void onLivePacket(IMediaStream stream, AMFPacket packet)
{
//WMSLoggerFactory.getLogger(CLASS).info("Packet Buffer " + packet.getData());
final Object lock = new Object();
synchronized(lock)
{
if (packet.isAudio()) {
try {
byte[] data = packet.getData();
ByteBuffer buffer = ByteBuffer.wrap(data);
WMSLoggerFactory.getLogger(CLASS).info("Audio Packet Buffer " + buffer);
//need input samplerate and channels
this.recorder.recordSamples(48000, 2, buffer);
//this.recorder.recordSamples(48000, 2, packet.getDataBuffer());
} catch (Exception e) {
e.printStackTrace();
WMSLoggerFactory.getLogger(CLASS).error("Error: " + e.getMessage());
}
}
}
}