Codec Wierdness Part One: the problem with proprietary media encoding

We’ve been working on an application that needs to capture the users voice using the local or client machine’s microphone. Once the sound is captured it needs to be converted into a WAV formated audio file.

Simple right?

Wrong. Very very very wrong.!!!!

We looked at Java implementations of JMF and decided for various reasons that Flash was the way to go. What those reasons were is the topic of another post and something I”d like to revisit in my own brain. It turned out to be a decision with consequences and the jury is out as to whether it was the correct one.

Flash is nice. Its always already installed on the user’s machine so its there and it affords a ready way to connect with the client microphone. That’s a big plus.

So all we have to do is to capture the voice stream and write it to an FLV file. Then we use FFmpeg which will trans-code almost all media files including FLV to trancode to WAV. Right?

The problem? [I'm quoting from another source]

Audio in FLV files is usually encoded as MP3. However, FLV files recorded from the user’s microphone uses the proprietary Nellymoser codec.

Encoding is done i client side, and f the lash player [doesn't] allow us to choose the codec . [We] have no way to record audio streams [using] other codecs.

Whatever RTMP server (FMS, Red5 or other one) the developer has no choice but to use Nellymoser when recording audio through the Flash Player.

And the problem is that they don’t sell or otherwise let you use a decoder without charing a fortune. It makes web development of medium to small project using the Flash voice recording capabilities nearly impossible. This is bad. Its bad for Adobe and its bad for this company — Nellmoser.

Someone will almost surely find a way to circumvent this problem and end up cutting them out of the loop. Adobe is surely could come up with something without relying on this codec and no doubt they will. The bottom line is that conversion of voice captured by Flash looked either too expensive, or too complex to be worth doing and JMF seemed to be viable solution.

However we came up with a Rube Goldberg system that seems to do the job. I do not recommend this approach. I’ll go into exactly what it is and other surrounding issues in a post tomorrow or next day.

More on the subject can be found below.

This guy has devoted his life to finding a workaround. He’s amassed quite a collection of sources which will give you insight into both the problem and possible solutions.