Flash Player H264 RTMP -> RTP

Wowza Streaming Engine
1

When publishing H264 using the flash player, and then streaming out with RTP I am seeing some very strange packets. Generally I see STAP-A packets which for the most part I am not expecting. Within these STAP-A packets are some unexpected and invalid NALU types. For instance 6, 9, 38. When publishing using FMLE I will only see NALUs of 1 or 28 (FU-A) (Or 1 and 5 once depacketized).

It is not so much the STAP-A packet that I am concerned with. It is the data in the STAP-A that does not correspond to a NALU of type 1 or 5 (I doubt a key frame would be placed in a STAP-A). On occasion our decoder will not like the NALU from the data in the STAP-A (Just ignoring the NALU type). On other occasions it is fine. I am guessing at this point that it could be AMF data in the RTP packets (the flash clients are using shared objects). I have not taken a hard look at the data in the NALUs that are invalid or ‘strange’. Generally they are very small (2 to say 24 bytes).

Also, if I ignore these packets then our decoder never works (only did a few tests). Any ideas you might have as to where to look, or how to approach looking at the RTP payload for issues would be appreciated.

– EDIT 1 –

I will have to do a better analysis. If I ignore STAP-A and dump any packets with less then 64 bytes it works fine. Obviously not an ideal solution. I will post back again with more information.

– EDIT 2 –

Here is what the data looks like.

All FU-A packet data is fine. The flash player seems to generate fairly large P frames which get placed in FU-A’s. The same for the key frames. Anything in the STAP-A packets is “garbage”. Is it possible that you are sending AMF/Shared Object data in STAP-A?

Thanks,

steve

2

6,7 and 9 Nalu type are normal. 6 and 7 are sps and pps units and 9 is the delimiter Nalu. Not sure about 38. Nalu 9 packets are 2 bytes. The other 2 are usually 4-20 bytes in length. So sounds normal.

Charlie

3

I am no longer receiving invalid types. The types I receive are 6, 9, 10, 5 and 1. These are all valid types. It is just very strange the way everything is ordered. A STAP-A it is almost always ordered 9, 6, 1. Other times 9 and 6 arrive in a single RTP packet. The P frames are sometimes in a STAP-A and other times in a FU-A. I do think it is with my code. I assume it is with my depacketizer and how I am handling the 9 and 10 types.

Actually 6 is a SEI. 7 and 8 are SPS and PPS.

4

This ended up being an issue where by the flash publisher changes it encoding parameters during it publishing session. Once this happens the SPS changes. Because the RTP subscriber is unaware of this the decoding starts to fail at this point.