[Buildbot-devel] Summer Of Code projects

[Nathaniel Smith]

On Sat, May 06, 2006 at 06:21:18PM -0700, Brian Warner wrote:
> * SQLifying the backend build-status database
> 
>  This would replace the current collection of directories and pickled Status
>  instances with a proper database, specifically one which could be
>  interrogated by external tools. I'd start with SQLite but it would be best
>  if other databases (MySQL and Postgres come to mind) could be swapped in
>  without too much effort. It has been suggested to me that divmod.axiom
>  should be used for the backend. Some considerable thought needs to be put
>  into getting the schema right, to make it useful to tool developers.

The idea that Axiom is a SQLite interface is very easy to get into,
since it does in fact provide an API for storing stuff in SQLite, and
it even sticks very close to the relational model.  But it's subtly
and horribly wrong.  Really, the way to think of Axiom is as yet
another try at tap/tac/... files, this time done _really_ right.

In particular, axiom databases are things that you can "start" (you
use "axiomatic start" instead of "twistd start", but that's basically
just because axiom isn't in twisted core).  This is subtly assumed in
the design -- e.g., the schema upgrade support works as an
axiom-provided IService.

Also, axiom databases are not really designed to be queried by other
tools.  The API presents a pythonic storage model with, e.g., stored
objects can contain references that can point to other stored objects
of arbitrary type.


However this is done, the functionality would be _really_ nice.  Our
server crashes every once in a while.  Whenever it does, I go and
rm -rf the buildbot directory (saving the config file) and then make a
fresh install.  Its database is invariably screwed up, and all builds
error out until I do this.  (It's a little more complicated, of
course, but this solution is by far the easiest.)


While I'm here, I'll also reiterate _my_ Number One Feature Request
(even though it's too late for SoC) -- some way for someone at the
master, to get a shell on a slave.  The whole point of buildbot is
that random people can with little effort donate some machine time
and then forget about it.  But, if anything changes in the build
environment we need, then I need to get all the people running
buildslaves to fix things up... and this is impossible, so in practice
we just lose 1/3rd or so of our slaves every time something changes.
It would be _much_ easier, and _much_ more effective, if I could just
go fix things myself without bothering anyone about it...

(There aren't any security concerns, anyone who can access the
buildmaster can already tell the buildslave to run arbitrary commands,
I'd just like a way to run those commands interactively.)

-- Nathaniel

-- 
Eternity is very long, especially towards the end.
  -- Woody Allen