[Buildbot-devel] schedulerdb progress report: week 1
warner at lothar.com
Mon Sep 14 19:19:14 UTC 2009
Just an update on what I've gotten done so far:
* http://github.com/warner/buildbot/tree/schedulerdb contains the new
code, branched off of 'master'
* "buildbot master" will create a sqlite-based database in
BASEDIR/state.sqlite , containing the new "changes" table (which
replaces the old changes.pck file.
* "buildbot upgrade-master" will create the same, then copy all of your
Changes out of changes.pck and store them in the database
* when ChangeSources hand changes to the buildmaster, they'll be stored
in the database
* rearranged buildbot/changes/changes.py to slowly get rid of the old
pickled classes: i.e. ChangeMaster is now a stub that exists solely
for pickle migration purposes, and a new ChangeManager class replaces
it. The Change class remains in place, since ChangeSources expect it
(they instantiate a Change and pass it to self.parent.addChange) as
well as for pickle migration.
* updated ChangeManager.getChangedNumbered and .eventGenerator to pull
from the DB
The new database schema is in buildbot/db.py . I wrote a wrapper around
twisted.enterprise.adbapi to permit synchronous access as well as the
usual Deferred-returning runQuery/runInteraction. This will make the
transition process easier (in particular, ChangeSources don't expect
addChange() to return a Deferred, so things work better if addChange()
blocks until the db write has finished). The long-term goal is to only
use async access. The Scheduler/Builder code that I'm writing will use
async access from the start.
Unit tests are currently broken, but "upgrade-master" should not throw
out your old changes (unlike the commit from earlier in the week), and a
basic test install correctly displays changes on the Waterfall.
Next steps are:
* test against mysql too (not just sqlite), test database-reconnection
code, think about how to provide database location to "buildbot
master" and "upgrade-master".
* update unit tests to match new classnames
* define schema for scheduler-state table, create notification loop,
start converting Schedulers to use the DB
BTW, the plan is for an 0.8.0 release to use the schedulerdb but be a
fairly painless upgrade from 0.7.11p3 . Only users who have written
custom Schedulers (and possibly, to a lesser extent, ChangeSources) will
need to modify their local code.
More information about the devel