<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
(didn't see the original message, sorry about that)<br>
<br>
Aakash,<br>
<br>
We use a very similar setup and for similar reasons. Our
single-master setup was bogging down completely. However, we elected
to not put any schedulers except our force schedulers into our ui
master. In our setup, every builder is tied to a particular master,
so the separation came naturally. We found it better for us to
duplicate essentially identical schedulers under different names
(each scheduler having the master that owns it as part of the name).<br>
<br>
We also use a single master.cfg to configure each master. Our
master.cfg uses basedir to determine which master is currently using
it (and uses '.' if none is, as when doing buildbot checkconfig .
for example). But ours is pretty complicated.<br>
<br>
We also have a script which will do a reconfig of all our masters at
the same time, and let us know if the reconfig was successful. That
way we don't have to worry about missing a master when we
reconfigure.<br>
<br>
One thing to keep tucked into your head is that if a master loses
its connection to the wamp router, you start getting some odd things
happening. But it happens so seldomly that it's hard to remember
that it might be a problem.<br>
<br>
Neil Gilmore<br>
<a class="moz-txt-link-abbreviated" href="mailto:ngilmore@grammatech.com">ngilmore@grammatech.com</a><br>
<br>
<div class="moz-cite-prefix">On 9/13/2018 3:15 PM, Pierre Tardy
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:CAJ+soVd1bmjzCPO5j39AWA=KPYY=je35VWvT8irfJ080VCC3ZA@mail.gmail.com">
<meta http-equiv="content-type" content="text/html; charset=utf-8">
<div dir="ltr">Hi Aakash,
<div><br>
</div>
<div>This sounds like a good plan.</div>
<div><br>
</div>
<div>What you can have is a haproxy distributing the workerload
over your two masters, this helps avoid to specifically
configure workers.</div>
<div>haproxy can work with roundrobin of tcp sockets that what
you should configure for your worker master.</div>
<div><br>
</div>
<div>Also, you should put your schedulers and changesource in
your UI master.</div>
<div>There are some bugs in the way buildbot do master election,
and you shouldn't count on it unless prepared to fix some bugs</div>
<div><br>
</div>
<div>You can restart the masters independently without issues
afaik.</div>
<div><br>
</div>
<div>Regards</div>
<div><br>
</div>
<div>Pierre</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr">On Wed, Sep 12, 2018 at 7:43 AM Aakash Jain <<a
href="mailto:aj355@cornell.edu" moz-do-not-send="true">aj355@cornell.edu</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">
<div dir="ltr">Hello,
<div><br>
</div>
<div>I have a multi-master Buildbot configuration with
following configuration:</div>
<div>master1: handling web-server,</div>
<div>master2: handling everything else (including workers
communication, build-request-distributor etc.)</div>
<div><br>
</div>
<div>Buildbot version 1.1.1</div>
<div>Twisted: 17.5.0</div>
<div>16-core cpu</div>
<div><br>
</div>
<div><br>
</div>
<div>However, master2 is probably heavily loaded and I
observe large delay in scheduling builds (even when
workers are idle) <a
href="https://github.com/buildbot/buildbot/issues/3661"
target="_blank" moz-do-not-send="true">https://github.com/buildbot/buildbot/issues/3661</a></div>
<div><br>
</div>
<div>I am planning to split master2 into two masters
(master2 and master3), one running on pb port 9989 and
another one on pb port 9990. Some of the workers would
be configured to connect to master2 and rest to master3.</div>
<div><br>
</div>
<div>All 3 masters would load same configuration. There
wouldn't be any load balancer, since all masters are
running on different ports (workers decide which master
to connect to based on how they are configured). I am
hoping that this would help with the delays, since both
master2 and master3 would be running the
build-request-distributor (activity_loop).</div>
<div><br>
</div>
<div>
<div>- Does this 3-master configuration looks correct?</div>
<div>- Is this a supported configuration?<br>
- Can I restart one master independent of other?</div>
<div>- Any feedback/comment/consideration?</div>
<div><br>
</div>
<div><br>
</div>
<div>Thanks</div>
</div>
</div>
</div>
<div dir="ltr">
<div dir="ltr">
<div>
<div>Aakash</div>
</div>
</div>
</div>
</blockquote>
</div>
-- <br>
<div dir="ltr" class="gmail_signature"
data-smartmail="gmail_signature">
<div dir="ltr">
<p dir="ltr"><br>
</p>
</div>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
devel mailing list
<a class="moz-txt-link-abbreviated" href="mailto:devel@buildbot.net">devel@buildbot.net</a>
<a class="moz-txt-link-freetext" href="https://lists.buildbot.net/mailman/listinfo/devel">https://lists.buildbot.net/mailman/listinfo/devel</a></pre>
</blockquote>
<br>
</body>
</html>