[Buildbot-devel] [1.0] Enough dreaming, let's get to work!

[Dustin J. Mitchell]

We now have a decent outline of Buildbot-1.0 at
  http://buildbot.net/trac/wiki/BuildbotOneOh
I think this is something we can accomplish within the span of the
next 2-3 Buildbot-0.* releases.

There are a few major design decisions left to be made.  At this
point, I'm looking for those who are willing to "walk the walk" --
potential developers -- to speak up.  Keep in mind that existing code
embodies a lot of hard work and lessons, so we should feel obliged to
keep it, to the extent that it is useful.

1. Overall Python framework.  Currently we use Twisted, and the most
deeply twisted part of the application is the job-control system,
which we seem to have agreed should be rewritten in a stateless
fashion.  Other frameworks (Django has been mentioned) would give us
better control over the web interface.

2. DB layer.  The main question seems to be the balance of difficulty
programming against a remote relational DB vs. the benefits in terms
of scalability, flexibility, and integration with external tools.

At this point, I think we should just *decide* and get down to coding.
 To that end, here are my opinions on these matters:

>From my limited understanding and from some tips on #django, mixing
Django with non-web processing is impossible -- we'd have to run a
separate Python process for the background operations (polling VC's,
communicating with buildslaves) and do some kind of RPC between that
and the frontend.  Given that, and the fact that we have a bunch of
perfectly functional twisted code, and an existing dependency on
twisted, I think we should just hang onto it.

As for the DB layer, I think we should stick with Sqlite for the
following reasons:
 - we won't be storing huge amounts of data (multiple GB)
 - most DB operations will be fairly simple queries
 - logfiles will be local files and not available via db queries anyway
 - axiom provides a lightweight, twisted-compatible ORM for it

Let's hear other opinions!  In the interim, I'm going to set up a git
repo to start hackin'.  More details on the repo once it's set up.

Dustin

-- 
Storage Software Engineer
http://www.zmanda.com