[users at bb.net] rc2 We have acheived multi-master. Mostly.

[Pierre Tardy]

Le jeu. 6 oct. 2016 à 18:44, Neil Gilmore <ngilmore at grammatech.com> a
écrit :

> Good morning everyone,
>
> This week I moved over to multi-master. It's kind of working.
>

Hi Neil,
That is good news!

For people interrested in multimaster, I have prepared a docker-compose
setup to demonstrate how to build a multimaster setup

It is available here:
https://github.com/buildbot/buildbot-docker-example-config/blob/master/multimaster/docker-compose.yml


I got a compatibility issue with latest autobahn though that I need to fix
first, I'm not sure if you saw it.


> We have 4 masters. One just has the UI and force schedulers (and the
> other schedulers, too). One has a particular category of workers. One
> has the other category of workers. And the last is supposed to have our
> process for monitoring results.
>
> It mostly works. Much of what doesn't is most likely my fault. For
> example, it turned out to be very much easier when porting our
> master.cfg to multi-master to decide whether to add a builder after
> creating it rather than before. Unfortunately, I seem to be adding
> MailNotifiers whether or not the builder is added. The result is that
> every failed build results in 3 emails.
>

indeed reporters are not a clustereredService like the scheduler are, while
they should probably be.
There is nothing that requires the reporter to be hosted on the same master
as the build.
So you can basically setup the email reporter on your UI maser or on your
monitoring master.


> That would have been merely comical, except that a process that nearly
> every builder uses (reporting stuff) crashed sometime this morning,
> which resulted in nearly every build failing (couldn't report). And
> other glitches in the conversion of master.cfg prevented the crashed
> process from getting restarted.
>
> At least I can fix that easily enough.
>
> We do get an awful lot of
> buildbot.db.schedulers.SchedulerAlreadyClaimedError. I'd left all the
> schedulers on all the masters as Pierre indicated to me that if one
> master fails, another will pick up the scheduler. But it seems as though
> this ought to be one of the exceptions suppressed by c['multiMaster'] =
> True.
>
Indeed, actually the code that dump the database error is generic, but
there are some errors that are normal in a multimaster setup.
Those are SchedulerAlreadyClaimedError, and BuilrequestAlreadyClaimedError.
This is a cosmetic issue that we should get fixed for 0.9.1.


>
> And congratulations on getting 0.9.0 released!
>
thanks! Now you should upgrade!

Pierre
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.buildbot.net/pipermail/users/attachments/20161006/efffdc19/attachment.html>