[users at bb.net] Migrating to Nine and our capability system

[Dan Kegel]

fwiw, I'm using a simple static capability system (the OS field in
https://github.com/buildbot/buildbot/blob/master/master/contrib/SimpleConfig.py
), and am thinking of extending it.  So I'm interested in the subject, too.
On Mar 23, 2016 5:27 AM, "Georges Racinet" <gracinet at anybox.fr> wrote:

> On 03/23/2016 01:07 PM, Pierre Tardy wrote:
>
>
>
> Le mar. 22 mars 2016 à 22:07, Georges Racinet <gracinet at anybox.fr> a
> écrit :
>
>> Hi there,
>>
>> After a long time during which we couldn't do much more than maintaining
>> our instances, we (Anybox) are considering migrating our buildbots to
>> Nine.
>>
>
> This is good news!
>
>
>>
>> Among the many things that I've developed for our needs [1], there is a
>> rather complete capability system [3], and hence I must decide whereas
>> it's worth porting it or not [4]. I know it's a fairly common need that
>> many
>> development shops have been doing in private or almost, but it's not
>> obvious to me what is freely available (for Nine) on this topic.
>>
>> http://trac.buildbot.net/ticket/3120 does not get to a conclusion, and I
>> couldn't find much on it in a quick search of recent messages on this
>> list. Maybe I missed something else ?
>>
>
> Since that, I figured Workers have properties, which can be used in
> NextSlave callback in order to match any capability.
> I think it would be much better to have something in the core, not
> depending on having the users writing a capability specific
> nextSlave/nextBuild callbacks.
>
> Actually, there are two aspect in the capability problem. The static and
> the dynamic.
>
> 1\ Static problem is the easiest. It looks this is what you implemented
> already (and metabuildbot has in a smaller extend):
> - One part of your config describe the workers, and their capabilities
> - Another part is describing the builders, and their required capability.
> The system will then automatically configure which worker to associate to
> which builders.
>
> Yes, that is exactly what I'm doing. It's a bit more actually, since our
> system also spawns builder variants (applying the same BuildFactory several
> times for different versions of some capabilities).
>
> The full capability description is stored as a dict-valued Worker
> property, something you may very well call a hack.
> Then at build time, a custom step extracts the needed properties for that
> build. This is good enough for us, but it is intrusive : the step must be
> introduced in the build sequence.
>
>
> 2\ The dynamic problem. Slave are chosen at the start build phase.
> - One part of your config describe the workers, and their capabilities
> - Another part is describing the builders, and their required capability.
> - Each build requests can require an additional set of capabilities, which
> can restrict more the set of slaves which can run them.
> In order to implement that efficiently, one will have to mess around
> inside the buildrequest distributor code, which I can understand looks
> scary.
>
> I just took a quick tour of that for an unrelated reason (
> http://trac.buildbot.net/ticket/3498), yes it's a bit convoluted.
>
>
> I am not sure which one you are needing/implementing.
> I think it is fine that you only implement 1/ if this is what you need,
> but I would like that the design opens the door to implementing 2 without
> need to rewrite everything.
>
> Interesting. I think I didn't know about nextWorker at the time I started
> this. What concrete use-case do you have in mind for 2/  ?
> Also, if the idea is that users don't need to implement
> nextWorker/nextBuild, then it means they may implement it for other
> purposes, and it follows that the dynamic dispatching must play nice with
> that, isn't it ?
>
> Regards,
>
> --
> Georges Racinet
> Anybox SAS, http://anybox.fr
> Téléphone: +33 6 51 32 07 27
> GPG: 0x33AB0A35, sur serveurs publics
>
>
> _______________________________________________
> users mailing list
> users at buildbot.net
> https://lists.buildbot.net/mailman/listinfo/users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.buildbot.net/pipermail/users/attachments/20160323/0c5a25fa/attachment.html>