Things have been quiet on this blog lately, but it’s not because I’m ignoring Feedwhip. On the contrary: I’ve been getting up early every morning and putting in some hours before switching over to the job that pays the bills.

Feedwhip’s big server died a few months ago, and I had to push the service onto a much less powerful server (from 2GB ram to 512MB and from two fast CPUs to one slower one). This had the expected effect of slowing down the service overall. Instead of running through all the feeds once per hour, it took upwards of four or five hours — even with user limits scaled way back.

Ever since that downgrade I’ve been working on performance issues. I drastically reduced the memory footprint, and rewrote a ton of the feed processing code to make it much more efficient. After several fits and starts, this weekend I finally started seeing this:

Feedwhip bandwidth graph

That’s a graph of the amount of bandwidth Feedwhip consumes. As you can see, it’s got a nice, steady, hourly rhythm to it. Previous incarnations would get stuck at a steady 100 kbps (far too low to be able to process everything in an hour), or slowly trail off over time (due to memory leaks or fatal errors). Now, though, I’ve got an awesome, steady signal. Seriously, I love that graph.

These performance improvements have taken a long time and been occasionally frustrating, so it’s really gratifying to see them finally pay off. The real beneficiaries, though, will be the end users. First of all, I’m now ready to throw the code onto a hosted server somewhere outside my basement — which means everybody will be able to get decent connection speeds and an improved user experience. And even more important, I can start working on a whole slew of new features that I know everyone is going to love.

Advertisements