Kevin Burton’s NEW FeedBlog

This week has been crazy. I’m under the gun to deliver an updated version of TailRank for Under the Radar so if I’m a bit behind on blogging I apologize.

I’ve had a lot of positive feedback on TailRank this week and just wanted to spend the time and follow up.

What I find most interesting out of this whole process is how ideas I had just a few months ago are already hitting mainstream and being reified into topics like “post grazing”, “attention engines”, and “reading lists”. Innovation is accelerating. I can’t remember this much happening back when we started Rojo.

Joshua Porter talks about information grazing (and what I’m most excited about with ‘post grazing’):

I would say post grazing is getting nearer our end goal. It is grazing for the latest, most interesting posts, regardless of what feed they come from. Right now, there are a few services out there putting out feeds with which we can post-graze, such as tech.memeorandum, Tailrank, and Findory. As I was telling Kevin Burton of Tailrank the other day, I’m completely in awe of folks who create these services: Gabe, Kevin, and Greg, in this case.

This is why I created TailRank. I want to tell the engine that I’m interested in a few topics and blogs and have it filter out most of the spam and just show me the “ham”. I think our initial release of personalized rankings became too confusing for most people but we’ve had great feedback. The next revision will be complete here shortly and I think a it will be a big win for everyone.

Alex Barnett thinks this will become a full blown attention race:

Your OPML file (specifically your list of RSS subcriptions) is one example of this Attention data set. It says alot about you: the topics your interested in and the people you listen to, and much more. There is plenty more Attention data that can be leveraged though. My tags, my wishlist, the books I own, etc.

We’re just at the beginning of the Attention Engine race.

Yes. I couldn’t agree more. It should be pretty exciting. We’re pushing hard and fast to get to the top of the pack and then clearly differentiate and provide better relevance (and gain a lap or two). Stay tuned… Good stuff is coming fast.

If personalization is where it’s at then it needs to scale (fortunately Danny (correctly) disagrees).

Not necessarily. Clustering and similarity algorithms are generally very computationally expensive, but the size of the source corpus is the same whether there’s 1 client or 100,000. Nik says “different indexes each time” – why not use the same indexes? Ok, I wouldn’t for a moment say it’s an easy problem, but like most it can almost certainly be broken down into manageable units.

There are ways to solve this problem. You just have to think out of the box.

Some of my competitors feel it’s too hard:

“I agree with Nik that there’s a huge technical chasm to cross before a “personalized meme tracker” gets really useful. I think progress I make on the memeorandum engine is approaching that, but it’s still far off enough that I’ll pass on hyping it for now.”

Fortunately I’ve already solved this problem. I have an extensive background in building scalable clustering technology (I spent most of my time doing this at Rojo) and our ranking algorithms scale linearly in large compute clusters. Scalability is a hard problem. There aren’t that many people who have ever run a Friendster, MySpace or Technorati.

Of course this isn’t just talk. We already have 90% of the functionality complete. You can go ahead and import your OPML into TailRank right now and build a personalized feed. You can even add this back into your aggregator if you want with a custom feed. We’re going to deploy additional functionality over the next week or two to really take it to the next level.

Because let’s face it, Personalization + Clustering is the next big step in RSS. If 2005 was about Aggregation, then 2006 is all about Filtering.

I totally agree Richard. I think this is one major issue that’s holding RSS back. RSS lead to an explosion of information (which is good) but its almost too good. There’s just too much to track. Which is why we need memetrackers.