[Buildbot-devel] CtuhuluBot (Was: Re: Fwd: [Buildbot] #641: New Default CSS)
macke at yar.nu
Sat Dec 12 10:52:31 UTC 2009
On 2009-12-10 14:44, Almad wrote:
> On Thursday 10 of December 2009 10:55:34 Marcus Lindblom wrote:
>> But having a separate web server and BuildBot as a web service backend
>> is an interesting idea. But I think we have a slight collision course
>> with Brians DB work there, as that will bring the data into a SQL DB,
>> from which several processes/threads can work.
> I'm actually working on that. My approach is:
> - django is used as main framework (this thing must first work inhouse, and
> we're djangonauts)
> - configuration is stored in rdbms, as it's convenient to use
> - build results are retrieved via status plugin and stored in mongodb
> database (via )
> - mongodb is used as it should be faster then rdbms, it's easier to work with
> (build results are json anyway). Asynchronous writes are also problem,
> compatible python library shold be on it's way 
FWIW, I'm a Django fan too :) but the buildbot's core data flows are
better suited for Twisted's async style. The web parts of twisted, OTOH,
are a bit clunky, IMO, but that's what we're working on.
(Mongodb looks cool, I'll have to dig into that a bit.. :)
> Project is called CthulhuBot and is available at . It's still WIP, but
> we're using it inhouse and it mostly works. However, it's not yet vcs-agnostic
> and friends (it's still kind of written for our git-based workflow) and just
> wrapping buildbot is not it goal.
> Additionally, there is simple "buildmaster client" cthylla  and set of
> "slave" setup.py plugins citools .
> My idea/hope is to enpower common users to create their own buildslaves as
> they need them, as well as create new project. So first thing on my plate is to
> have better configuration (so you write a plugin for company that says "create
> debian package according to our policy and upload it to repository, and user
> only says on web "OK, I want this for my project". This works now), but I'm
> assuming company-like network (= master has ssh access to slaves).
Interesting. Thinking further, a public repository of web-configurable
plugins would be cool, as you currently have to be quite python savvy to
set up a build master. (and there doesn't seem to be that much sharing
of custom build steps, which I think would be a good thing to make it
obvious that buildbot can be pretty useful for more things than just
reporting fail/success and logging stdio)
> Second plan is to make better reporting, I'd like to have "developer
> dashboard" displaying all commits done by logged user and their test results
> (kind of filtered tgrid).
Aye. We need to make the web interface more accessible and extendable. I
hope this will be possible if/when the data goes into SQL, making the
"clients" of said database language/library/buildbot agnostic. I.e.
empower users to write clients that do their favourite stuff in their
> Also, RSS/pubsubhubhub should be super easy by then.
> I'm now in phase "first step: it's crufty, but works for us", so I'm now
> writing docs and preparing to polish things and stuff. My thoughts were to
> announce project when I'll have neat tutorial "how to automatically build your
> python package in virtualenv with few clicks" ;)
That'd be cool. :)
> Comments welcomed, althrough we should probably move to separate topic/thread.
>  http://github.com/ella/buildbot-mongodb-status
>  http://github.com/fiorix/mongo-async-python-driver
>  http://github.com/ella/cthulhubot
>  http://github.com/ella/cthylla
>  http://github.com/ella/citools
More information about the devel