So yea. It works like this:

You go to the page and provided you have Flash enabled (you do or you won’t be reading this to begin with) click record and this pops up.

The file is recorded and that’s it.

Seems simple right?

Its not.

When I have time later I’ll go into the details.

See this for the configuration options.

We’re designing another topic based publishing system for a client who wanted to know how news items are moderated. In other words how does the system pick and chose what the readers see?

Will the news pick up articles like this? Do we have to “crawl” the [omitted] site for this stuff?

http://www.blahblahba.com/article/email/idUSN22467289xyz

ANSWER:

Generally ‘crawling’ is where you have bots or your machine agent go from one site to the next and follow links based on predefined criteria or as Google calls it “an algorithm.’ Its just like wondering the net and taking notes on everything.

So while we can do it in theory we aren’t going to ‘crawl’ anything. You don’t want to do that as it is both resource intensive and ineffective unless you have the resources.

The answer to your question however is ‘yes’ and ‘no’. The following is the only coherent answer I can give you that is true and I know right now. I know allot about this but not as much as some others.

First it will aggregate EVERYTHING we specifically tell it to or any feed we put in the system will be aggregated. So if blah—all of blah —is aggregating then of course this feed item your pointing to will be one of them. However as I said there is a filter system. You do not want every feed published even if it is aggregated and in the database. That’s pointless as you know. You only want feeds published that are either topic based and you subscribe to on purpose because it is written specifically for your subject OR…. You want individual feed items [ITEMS ARE NOT FEEDS]published and promoted based on filters ]. The items are scanned by a filter system that you/we/your designers are going to have to setup and maintain all the time. The filter system is only good if you work it / refine the results… and not all that much different than what Google does. Actually its exactly what Google does.

It works like this. The filter system looks at a pool of x feed ITEMS , and scans for key words or phrases. Then when it finds a hit it can do one of three things .. 1. publish the item or 2. publish and put in the moderation queue for a human editor to approve or not or 3. publish AND PROMOTE the item to the front.

So to recap. You can setup certain feeds where all the feed ITEMS are promoted and published. I would call these your featured newsfeeds. You can also subscribe to non topic or general feeds and apply the filter which will help you pick out the items you want to promote to the front. It can also just promote them. It can also be set to promote the items from some feeds or trusted feeds and queue others for moderation. Etc.. ..

FYI it works well. I’ve setup sites as experiments that don’t use the human editor part (I don’t advise this btw) and it more or less works but pick out the feeds that are relevant. Adding the human is the key here and one person working 30 minutes every other day can do all that’s needed to make the site contain tons of very relevant, non Spam, content. You should read that blog post where I answered the charge that I was a splogger. That’s me who wrote it and the so called review of the system’s effectiveness was not something I solicited but under the circumstances I decided to go with it. FYI this was before we had any kind of filter system designed.

I hope this helps you understand how it works.

Now there is stuff I can’t really explain right now. As I’ve said I need to get someone to begin the work on the podcasting and feed aggregation system to make the first and refine the second. I also want know we’re  continuing to refine the look and feel based on your original design but are going to be adding panels and other theme/content systems to create a series of feed blocks.

We’ve been working on an application that needs to capture the users voice using the local or client machine’s microphone. Once the sound is captured it needs to be converted into a WAV formated audio file.

Simple right?

Wrong. Very very very wrong.!!!!

We looked at Java implementations of JMF and decided for various reasons that Flash was the way to go. What those reasons were is the topic of another post and something I”d like to revisit in my own brain. It turned out to be a decision with consequences and the jury is out as to whether it was the correct one.

Flash is nice. Its always already installed on the user’s machine so its there and it affords a ready way to connect with the client microphone. That’s a big plus.

So all we have to do is to capture the voice stream and write it to an FLV file. Then we use FFmpeg which will trans-code almost all media files including FLV to trancode to WAV. Right?

The problem? [I'm quoting from another source]

Audio in FLV files is usually encoded as MP3. However, FLV files recorded from the user’s microphone uses the proprietary Nellymoser codec.

Encoding is done i client side, and f the lash player [doesn't] allow us to choose the codec . [We] have no way to record audio streams [using] other codecs.

Whatever RTMP server (FMS, Red5 or other one) the developer has no choice but to use Nellymoser when recording audio through the Flash Player.

And the problem is that they don’t sell or otherwise let you use a decoder without charing a fortune. It makes web development of medium to small project using the Flash voice recording capabilities nearly impossible. This is bad. Its bad for Adobe and its bad for this company — Nellmoser.

Someone will almost surely find a way to circumvent this problem and end up cutting them out of the loop. Adobe is surely could come up with something without relying on this codec and no doubt they will. The bottom line is that conversion of voice captured by Flash looked either too expensive, or too complex to be worth doing and JMF seemed to be viable solution.

However we came up with a Rube Goldberg system that seems to do the job. I do not recommend this approach. I’ll go into exactly what it is and other surrounding issues in a post tomorrow or next day.

More on the subject can be found below.

This guy has devoted his life to finding a workaround. He’s amassed quite a collection of sources which will give you insight into both the problem and possible solutions.

We’re building a publishing system for a client whose interested in creating a podcast directory.

In the course of things the question came up about subscribing to a custom feed generation service that in effect acts as a topic aggregation system and generates feeds based on your choice of topic.

I don’t this its needed here because we already have a filter system that will scan feeds and only promote those that meet certain key words and such that you can set. This service is not much different really.

Our system is dependant on the administrators or publisher actually tuning the keywords and looking at the results. It only gets better the more you use it or the more you figure out what those key words are.

Let me clarify this…. I think a third party service can be useful for this kind of topic feed but it should augment our built system and really only after you know what your needs but it isn’t something we’ve not already developed as well.

Remember that we’ve been working on a ‘publishing system’ all along. More specifically it is an automatic system that is topic focused but designed to create human readable web pages. The human readable part is key. It creates the illusion of having the content edited even when there is no editor. Part of this illusion is to be sure the content promoted to the front is relevant. So in order to work the system has had to be sure that feed items meet some criteria OR be from a very select group of feeds OR both.

An unintended side benefit of creating the system with human readability in mind is that it gets a high search profile. I.E. the things one does to make it palatable to people is exactly what the search engines deem as relevant.

This is not my plea to free Bernie Ebbers or to posthumously pardon Ken Lay. The latter may rest in some peace however knowing that he didn’t just preside over one of the largest acts of corporate thievery ever but helped lay (multiple puns) the groundwork for many of technological advances that are the hallmark of web 2.0. The general availability of upskirt video and Youtube will forever be one of the lasting achievements of the previous decade. These things will be — already enjoyed by college student worlwide — recognized as being an advance in productivity that was the result of the fiber optic gold rush of the late 90’s.

I got to enjoy some of the rush (about 3 months) before the damn broke which swept us all out to sea in the reverse flow. I’d been involved in a startup that was one of the first attempts to commercialize MPEG 4 over standard broadband. We had some remarkable success in those early days and its a subject for another post or series of posts.

The point is that we were funded by a fiber network needing some visual spice to perk up the dog and pony show it had to provide its potential investors. We were both the dog and pony and it worked to a large extent. When every any of the unconvinced fund managers or VCs would pipe up to the effect that maybe there was no market for all the bandwidth they would whip out the old laptop and show the nearly DVD quality encoded Matrix that became our Hallmark. (Warner was not amused btw and we took it down).

They — the company — were able to raise 10s of millions which was relative chicken feed up against the hundreds of billions being lavished on the fiber companies in anticipation of the broadband revolution. That revolution turned out to be more of an evolution thus helping to drive many of the players from the field when the revenues didn’t materialze before the damn broke open and the pool of cash drained into the war in Iraq.

However the unheralded legacy can be quickly seen by looking at the snapshot graphic above. Its just one of the many fiber networks feeding into my new favorite server provider – Softlayer. I admit this is unscientific but its a good indication of the level of surplus in bandwidth left by the 90s goldrush in the telecommunications fiber optic business.

We’ve had to devise a way for a client to quickly encode multiple sources of high quality video using white box equipment. The video would come in spurts from webcam captures and uploaded WMV and AVI.

The problem in this instance is not that their server can’t take the load. It easily could . Well tuned video applications will sense the server load and queue the video encoding jobs. Somehow Youtube used to do this very thing in its early days.

They — the client — absolutely didn’t want to ramp up to the slightly more expensive dual CPU class as we suggested initially. Their point which I incorrectly dismissed was the dual processor would have collateral issues that were not worth the possible benefits. It would mean tuning or even changing much of their server software and siphon off an unacceptable level of management focus. Not to mention consulting and IT expense. This is true actually. No matter how well planed out moving to new network hardware is alway s fraught with unforeseen problem.

The solution is simple on paper.

We can insert a relatively inexpensive graphics card and pipe much of the heavy lifting through the GPU with FFmpeg. The testing on the local server shows truly amazing results. Properly configured a plain Jane Linux box can be a real video workhorse.

I’ll post the hardware details after a few more days of testing and talks with the hosting provider are done..

We’ve been having issues with our ability to use PNG graphics as the source for a logo on the Flash videos. Take a look at this.

The problem is that using PNG doesn’t work and the Jpeg doesn’t allow alpha transparencies. We need to get rid of the white box and have some approaches but its an example of the little details that will drive you crazy .

Back in 2000 we started a contest to give an iMac to the first person / group to come up with a video container or method that would allow for streaming of Divx encoded Mpeg4 video over broadband. At that time there was no such method and Divx Networks — the company — didn’t exist.

We were able to sucessfully stream Divx files and some code landed in our laps as a result of the contest. That code was the first beginnings of h264.

From Slashdot today.

“According to Bloomberg, a jury ruled against Qualcomm in their patent lawsuit against Broadcom. Qualcomm had sought $8.3 million in damages for patent infringement stemming from Broadcom’s H.264 encoder/decoder chips. From the article: ‘The patents, covering a way to compress high-definition video, are unenforceable in part because Qualcomm withheld information from the U.S. Patent and Trademark Office, jurors in San Diego said today after deliberating less than six hours.’ This ruling clears the way for H.264 to become a widely adopted open standard.

Some news on the codec front [also from the Slashdot piece]

Already Linux and Free software people have a good H.264 implimentation thanks to the FFmpeg people. Their mpeg4 Divx-stuff is already very high quality.. much better then anything from Xvid or Divx, they have the beginnings of very good H.264 support and have decoding and encoding speeds that rival the best propriatory [sic] codecs [sic] aviable. They need to fill out some of the H.264 features, but if this is true that H.264 is truly usable in Free software environment, then I expect that development will very quickly take off as the people become aware of this and Linux distros will want to jump on the opportunity to provide world-class HD support!

All of this makes for some interesting possibilities to my way of thinking. Quicktime is a far better transport and delivery system for web video than Flash. There. I said it.

With the addition of H264 Quicktime and Apple have laid the ground word to become a serious and better alternative to high quality video streaming.

I’m going to offer some reasons in the part two of this piece.