Paul Swartz's PCF Devblog

The last week was mostly a bug-fixing week: I think I got through at least 10 bugs on Bugzilla for the Guide. I added the Top Rated channels page, and changed the layout for that and the popular channels page to show the rating.
I did a little bit of caching work, and the site is feeling a lot faster. Last week, the worst page had an average rendering time of 20 seconds. Now, the worst rendering time is 3 seconds, and the average time is 0.6 seconds. I’m still working on it with BDK to make it a little simpler on our end, but not at the expense of the caching being effective.
This week I’ll be working on adding some features: releasing approved channels slowly to avoid overloading the “New Channels” page, the ability to download individual videos from channels, and some simple statistics for channel owners.

(This is simply the status e-mail I sent to the rest of the developers this evening.)
Earlier this week, I did a bit of work fixing small details on the new ratings page. However, I’ve mostly been working on getting Miro Guide to be more cache-friendly. The current hit rate is roughly 20%, up from a measly 14% last week. I think this can be much better, however.
There are two general ways to use the cache. One is to cached rendered templates, the other is to cache database queries. Both I think are important, but I’ve been focusing more on the database end of things. At that level, it’s easier to tell what objects/tables are being modified.
Monday and Tuesday I was working on implementing a List class which represents a database query that is cached in-memory and updated when inserts/updates/deletes happen in the database. However, I’m running in to problems with my current approach. Because there’s no master list of queries, there’s no way to make sure every list is updated when it should be. I came up with a new idea on the bus, which is to make each cached query have an id# which is incremented when the table is modified. This will invalidate all the old cache keys and make it hit the database to reload the cache. For queries like the popular channels, we can not invalidate the keys on table updates, but simply on time; this will keep the cached data around for a rough amount of time (say a minute or 5) before hitting the database again.
I’ll be working on implementing this idea tomorrow. What this can’t do, unfortunately, is update the cache at the same time the database is updated. I can do that for individual records (I have a nice CachedRecord class which does this, and I’ll start using it around the code base) but that doesn’t work for database queries. I plan to do a bit of research as well to see if other people have perhaps tried to solve this problem.