[Buildbot-devel] Going beyond the tutorial; ideas for simplifying larger configurations

[Pierre Tardy]

This actually quite interesting to read all those thoughts, and usage of
buildbot, I can understand how buildbot is really used in many different
way.

That is the power and the weakness of buildbot. The tool lets you be very
creative on how you can implement your process, and this allows you to
create very powerfull stuff based on your exact need.

It is quite easy for beginner people to get frustrated by that. Other open
source and non opensource CI tools are in the big majority giving a nice UI
to describe your process. I've tried some proprietary Ci system recently,
and I was amazed how you can setup master, slave, configure git repo,
trigger build on commit, they have a UI for conditionnal trigger were you
can describe the conditions entirely with mouse and dropdown menus like in
Google App Inventor...

Can we do that with buildbot ? For sure!
I think that the buildbot-travis project is actually a good step forward,
the only needed thing then is to have a web UI that will allow to display
and edit the yaml files. Then the bootstrap scripts could create a generic
master.cfg that would only contain a
c = TravisConfigFactory()

Simplifying user experience for simple stuff will probably help us to gain
market share. Once people have succesfully used buildbot for their simple
project, they can dive into the marvels of python-script-as-config.
When you talk with people using Jenkins for complex projects they often
tell that the web UI make it very hard to maintain complex stuff.
At the time when they choose their CI system, they probably did not know
exactly what CI means, and their requirements were very simple, so that
Jenkins was very appropriate, and they were happy to setup the whole thing
in half a day. But then their needs becomes more and more complex, and it
is too late to move to a more flexible system

When I was choosing what I could use for my build automation problems, I
read this blog article, and understood how much python-script-as-config is
powerful:
http://corp.dazzit.com/technology-blog/buildbot-command-control-communication-part-2.html

Now, at work we use the build_factory(args) idea very heavily, and we
actually load the 'args' dictionaries from a group of yaml file, so that we
separate in the config what is the logic of our process (coded in python),
and what is the configuration of the process (git urls, branch names, user
rights, etc, described in yaml files)
If we have UI framework to visually edit travis yaml file, probably it
would be easy to also implement editor for such yaml config files.
The overall goal is to have a generic master.cfg that implements a
"gated-commit" CI process with Buildbot, Repo, Gerrit and Bugzilla (and
then probably open-source it)

Regards,
Pierre


On Tue, Aug 21, 2012 at 8:59 AM, Dan Kegel <dank at kegel.com> wrote:

> The buildbot manual is good - better than, say, the cmake manual -
> but it's still hard for beginners when faced with largish buildbot
> configurations to make sense of them.
> The tutorial helps, but it stops at a very simple configuration.
>
> I just joined a group that maintains eight or so related
> projects (one core library, and seven apps that use it).
> They use one buildbot for each of the projects.
> Over time, evidently the configuration got unruly,
> so they wrote a central configuration library that did the
> bulk of the work for master.cfg.
> Adding a new project to the central config is
> difficult because that central configuration ilbrary
> needs changes in about eight different areas to add
> one new project.
> Also, there was no simple script that automated
> creating/installing the many masters and slaves.
> (As I whittle away at these problems, I get the feeling that
> I could take what I'm learning and do a larger example
> for the tutorial.  No promises, though.)
>
> Before I dive in and turn our central config library inside out,
> has anyone got advice about structuring medium-sized
> buildbot configurations?  I've been looking at the Chromium buildbot
> configuration for ideas
> http://www.chromium.org/developers/testing/chromium-build-infrastructure
> For instance,
> http://lackingrhoticity.blogspot.com/2011/08/fixing-trouble-with-buildbot.html
> describes one of their ideas.
> Any other good sources?
> - Dan
>
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Buildbot-devel mailing list
> Buildbot-devel at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/buildbot-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://buildbot.net/pipermail/devel/attachments/20120821/9b7dfa26/attachment.html>