[Buildbot-devel] New, more efficient web stuff

[Jean-Paul Calderone]

On Fri, 4 Jan 2008 12:21:56 -0500, "Dustin J. Mitchell" <dustin at zmanda.com> wrote:
>On Jan 4, 2008 12:04 PM, Jean-Paul Calderone <exarkun at divmod.com> wrote:
>> What makes SQLite-only a non-starter (I don't see how this relates to
>> Twisted version, either)?
>
>My installation of buildbot, at least, has >2.3G of data, and in my
>experience SQLite doesn't do well over ~1G.
>

SQLite doesn't have the best query optimizer in the world.  If you give
it something it can't whittle down to sub-linear, then you'll definitely
feel it when your database gets large.  However, if you have the right
indexes, then it's quite fast.  I have a 5GB SQLite database that basically
performs well enough.

I don't think buildbot needs a particularly complex or nuanced data model.
I expect it is possible to express all of its requirements in terms of log
N or queries.

>> If that's really the case, then you can look at Storm, which has a lot
>> in common with Axiom, but supports PostgreSQL and MySQL and various other
>> things (as well as SQLite).
>
>You still didn't answer the question, though -- why should I use
>something new and untested, when all existing buildbot users already
>have something that works?

Hopefully my other reply addresses that to some degree.

>
>> Indeed. :)  I think making it possible isn't very much work.  You just need
>> to insinuate guard into the resource hierarchy and then make the resource
>> that you have the avatar for the anonymous user.  As developers have
>> interest and time, they can create non-anonymous avatars with functionality
>> that should be restrict and use various flags to disable functionality for
>> the anonymous user.
>
>I've never understood guards, since they're not in the twisted
>tutorial.  Do you have a link that can explain how to use it?

Hmm.  I don't know of any good documentation for guard, offhand.  There may
not be any. :/  There is some cred documentation, not sure if you've read
that already or if you're otherwise familiar with cred.  If not, that's a
place to start at least.  Guard is basically just the HTTP integration for
cred.

Jean-Paul