<html>
<head>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">On 03/27/2016 05:35 PM, Georges Racinet
wrote:<br>
</div>
<blockquote cite="mid:56F7FDB7.7070602@anybox.fr" type="cite">
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
<div class="moz-cite-prefix">I have now evidence that our
capability system works with Nine (with actually almost no
change).<br>
Now for our own sake, I will separate from our actual build
definitions in a git repo of its own, and see if I can make nice
examples based on meta buidbot and SimpleConfig use-cases.<br>
</div>
</blockquote>
So, here it is :
<a class="moz-txt-link-freetext" href="https://github.com/anybox/anybox.buildbot.capability">https://github.com/anybox/anybox.buildbot.capability</a> (please, don't
bother about the name itself, it's just an easy way for me make sure
it won't conflict with anything else).<br>
<br>
As it's been said before, it's a system to declare "capabilities" on
workers, and spawn automatically builders according to requirements
or multiplication (build_for) directives. This is "static" because
this happens at buildmaster startup or reconfiguration time. There's
a full explanation at
<a class="moz-txt-link-freetext" href="http://docs.anybox.fr/anybox.buildbot.capability/master">http://docs.anybox.fr/anybox.buildbot.capability/master</a><br>
<br>
It's now in production on Anybox's buildbots for public projects
(can't show until I've found a safe way to re-enable dual anonymous
& auth access).<br>
<br>
Now, we could either (with increasing preference on my part)<br>
<br>
- include it in contrib/<br>
- leave it and publish on PyPI as-is<br>
- publish it separately on PyPI as maybe buildbot_capability ?<br>
- include it in the core (Pierre seemed to favor that, but how
precisely ?)<br>
<br>
I don't favor inclusion in contrib because if I understand it
correctly, its tests would not enter the general buildbot test suite
(and it has very good coverage).<br>
<br>
Also, I've taken a look at contrib/SimpleConfig.py, but its examples
don't seem to be a good match to display the multiplication feature,
because the build variants are based on arbitrary tags, not on
versions (there's an example with workers having two Ubuntu
versions, but no build uses both of them if I got it right). Those
examples could be rewritten using the filtering requirements, but
that actually makes it less simple.<br>
On the other hand, this looks closer to what meta-buildbot might
need (looked really too quick to be sure)<br>
<br>
Regards,<br>
<br>
<blockquote cite="mid:56F7FDB7.7070602@anybox.fr" type="cite">
<div class="moz-cite-prefix"> <br>
Pierre, I don't forget the dynamic variant, it's just simply
farther away, given what we already have.<br>
<br>
Regards,<br>
<br>
On 03/23/2016 03:39 PM, Dan Kegel wrote:<br>
</div>
<blockquote
cite="mid:CAPF-yOb+VFS8DgOgaaB8amvMNaSL-5pnxdghRTNrr6GQOZ1wBA@mail.gmail.com"
type="cite">
<p dir="ltr">fwiw, I'm using a simple static capability system
(the OS field in <a moz-do-not-send="true"
href="https://github.com/buildbot/buildbot/blob/master/master/contrib/SimpleConfig.py">https://github.com/buildbot/buildbot/blob/master/master/contrib/SimpleConfig.py</a>
), and am thinking of extending it. So I'm interested in the
subject, too.</p>
<div class="gmail_quote">On Mar 23, 2016 5:27 AM, "Georges
Racinet" <<a moz-do-not-send="true"
href="mailto:gracinet@anybox.fr">gracinet@anybox.fr</a>>
wrote:<br type="attribution">
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<div>On 03/23/2016 01:07 PM, Pierre Tardy wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr"><br>
<br>
<div class="gmail_quote">
<div dir="ltr">Le mar. 22 mars 2016 à 22:07, Georges
Racinet <<a moz-do-not-send="true"
href="mailto:gracinet@anybox.fr" target="_blank">gracinet@anybox.fr</a>>
a écrit :<br>
</div>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">Hi
there,<br>
<br>
After a long time during which we couldn't do much
more than maintaining<br>
our instances, we (Anybox) are considering
migrating our buildbots to Nine.<br>
</blockquote>
<div><br>
</div>
<div>This is good news!</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
Among the many things that I've developed for our
needs [1], there is a<br>
rather complete capability system [3], and hence I
must decide whereas<br>
it's worth porting it or not [4]. I know it's a
fairly common need that many<br>
development shops have been doing in private or
almost, but it's not<br>
obvious to me what is freely available (for Nine)
on this topic.<br>
<br>
<a moz-do-not-send="true"
href="http://trac.buildbot.net/ticket/3120"
rel="noreferrer" target="_blank">http://trac.buildbot.net/ticket/3120</a>
does not get to a conclusion, and I<br>
couldn't find much on it in a quick search of
recent messages on this<br>
list. Maybe I missed something else ?<br>
</blockquote>
<div><br>
</div>
<div>Since that, I figured Workers have properties,
which can be used in NextSlave callback in order
to match any capability.<br>
</div>
<div>I think it would be much better to have
something in the core, not depending on having the
users writing a capability specific
nextSlave/nextBuild callbacks.</div>
<div><br>
</div>
<div>Actually, there are two aspect in the
capability problem. The static and the dynamic.</div>
<div><br>
</div>
<div>1\ Static problem is the easiest. It looks this
is what you implemented already (and metabuildbot
has in a smaller extend):</div>
<div>- One part of your config describe the workers,
and their capabilities</div>
<div>- Another part is describing the builders, and
their required capability.</div>
<div>The system will then automatically configure
which worker to associate to which builders.</div>
</div>
</div>
</blockquote>
Yes, that is exactly what I'm doing. It's a bit more
actually, since our system also spawns builder variants
(applying the same BuildFactory several times for
different versions of some capabilities). <br>
<br>
The full capability description is stored as a dict-valued
Worker property, something you may very well call a hack.<br>
Then at build time, a custom step extracts the needed
properties for that build. This is good enough for us, but
it is intrusive : the step must be introduced in the build
sequence.<br>
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_quote">
<div><br>
</div>
<div>2\ The dynamic problem. Slave are chosen at the
start build phase.</div>
<div>
<div>- One part of your config describe the
workers, and their capabilities</div>
</div>
<div>
<div>- Another part is describing the builders,
and their required capability.</div>
</div>
<div>- Each build requests can require an additional
set of capabilities, which can restrict more the
set of slaves which can run them.</div>
<div>In order to implement that efficiently, one
will have to mess around inside the buildrequest
distributor code, which I can understand looks
scary.</div>
</div>
</div>
</blockquote>
I just took a quick tour of that for an unrelated reason (<a
moz-do-not-send="true" class="moz-txt-link-freetext"
href="http://trac.buildbot.net/ticket/3498"><a class="moz-txt-link-freetext" href="http://trac.buildbot.net/ticket/3498">http://trac.buildbot.net/ticket/3498</a></a>),
yes it's a bit convoluted.<br>
<br>
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_quote">
<div><br>
</div>
<div>I am not sure which one you are
needing/implementing.</div>
<div>I think it is fine that you only implement 1/
if this is what you need, but I would like that
the design opens the door to implementing 2
without need to rewrite everything.</div>
</div>
</div>
</blockquote>
Interesting. I think I didn't know about nextWorker at the
time I started this. What concrete use-case do you have in
mind for 2/ ? <br>
Also, if the idea is that users don't need to implement
nextWorker/nextBuild, then it means they may implement it
for other purposes, and it follows that the dynamic
dispatching must play nice with that, isn't it ?<br>
<br>
Regards, <br>
<br>
<pre cols="72">--
Georges Racinet
Anybox SAS, <a moz-do-not-send="true" href="http://anybox.fr" target="_blank">http://anybox.fr</a>
Téléphone: <a moz-do-not-send="true" href="tel:%2B33%206%2051%2032%2007%2027" value="+33651320727" target="_blank">+33 6 51 32 07 27</a>
GPG: 0x33AB0A35, sur serveurs publics
</pre>
</div>
<br>
_______________________________________________<br>
users mailing list<br>
<a moz-do-not-send="true" href="mailto:users@buildbot.net">users@buildbot.net</a><br>
<a moz-do-not-send="true"
href="https://lists.buildbot.net/mailman/listinfo/users"
rel="noreferrer" target="_blank">https://lists.buildbot.net/mailman/listinfo/users</a><br>
</blockquote>
</div>
</blockquote>
<br>
<br>
<pre class="moz-signature" cols="72">--
Georges Racinet
Anybox SAS, <a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://anybox.fr">http://anybox.fr</a>
Téléphone: +33 6 51 32 07 27
GPG: 0x33AB0A35, sur serveurs publics
</pre>
</blockquote>
<br>
<br>
<pre class="moz-signature" cols="72">--
Georges Racinet
Anybox SAS, <a class="moz-txt-link-freetext" href="http://anybox.fr">http://anybox.fr</a>
Téléphone: +33 6 51 32 07 27
GPG: 0x33AB0A35, sur serveurs publics
</pre>
</body>
</html>