[Buildbot-devel] New, more efficient web stuff

[Dustin J. Mitchell]

On Jan 5, 2008 4:10 AM, Nathaniel Smith <njs at pobox.com> wrote:
> The justification is just that sqlite is vastly simpler to deal with
> than any other particular database, and *any* particular database is
> *vastly* simpler to deal with than a "generic database backend".  Just
> say no to premature architecture.

This is definitely a fair assessment.

> Note also BTW that for 2.3G of data you will surely will have far, far
> fewer than 2.3 billion *rows*; I guess the vast majority of that 2.3G
> is build logs, which would be stored in big blobs in the database and
> sqlite wouldn't even touch those pages when doing anything
> query-related.  The important question is something like, how many
> total builds (probably more like "thousands" than "billions") do you
> want to keep track of, and can sqlite scale to *that*?

This is also true, and furthermore I don't intend to put the logs in
the db (any sort of "tail" operation would be vastly inefficient that
way).

I'm now convinced that SQLite should be the first storage backend I
write, although the door remains open to adding others :)

Shaun -- build properties are stored as well, no worries.

Dustin

-- 
Storage Software Engineer
http://www.zmanda.com