By far the most database-intensive thing Feedwhip does is look for feed items. Every change to a feed gets stored as a feed item, even tiny little changes to comment counters, and so finding the changes you care about for a given feed can be really slow. Some of the RSS feeds take hundreds of seconds to generate. Wow, that’s slow.

RSS feeds generally don’t change as often as people poll them for changes. This means that I keep looking up and regenerating the same page over and over again. The solution, of course, is to cache the changes on the web server so that we don’t need to hit the database and grind through the code unnecessarily. This point gets mentioned over and over again when people talk about how something as slow as PHP (see my previous post) can power a site as popular as Digg.

Of all the perf changes I’ve done (or have planned), I think this one will probably have the biggest impact. Which makes me wonder why I’ve waited so long to do it…