[Buildbot-devel] database-backed status/scheduler-state project

[Dustin J. Mitchell]

On Thu, Sep 3, 2009 at 10:26 AM, <exarkun at twistedmatrix.com> wrote:
> I'm curious about the motivation to use the database directly as the RPC
> mechanism here.  This makes the database schema part of the public API,
> something which impedes future improvements to the schema.  Also, if the
> data's constraints aren't *entirely* represented in the database schema,
> it opens up the possibility for invariants to be broken by data
> incautiously inserted by other tools.

Having written a Python daemon that's controlled from Access via ODBC
and various silly "control" tables, I heartily agree.  Besides the
reasons above, you also commit the daemon to "polling" the database
for changes to the control tables, or (horrors!) writing DB triggers
that somehow signal the daemon that something has changed.  The
polling leads to a delay, which means that things are often
inconsistent for a few seconds from the user's perspective, leading to
spurious 404's and other such annoyances.

There are pretty clear "right" ways to implement all of this (I'm
thinking of plans to use synchronous DB queries here as "wrong"). The
problem we've encountered in discussions like this over the last year
is that it's hard to see what the relative costs are to implementing
it "wrong" in the interests of speed and minimizing changes.  Will we
end up with a frankenstinian amalgam that experiences strange
slowdowns and in which users have to be instructed to "click slowly"?
Will the underlying code be maintainable?

On the other hand, from the "One-Oh" side of things -- Will the new
version ever get finished (or started)?  Will there be enough
similarity that users of the old version will be willing to migrate to
the new?  Etc.

I don't say all of this to imply your task is impossible, Brian --
only that we should all think of the long-term impact of these
decisions.

-- 
Open Source Storage Engineer
http://www.zmanda.com