[users at bb.net] Migrating to Nine and our capability system

Sun Mar 27 15:35:19 UTC 2016

Thank you for the interest,

I have now evidence that our capability system works with Nine (with
actually almost no change).
Now for our own sake, I will separate from our actual build definitions
in a git repo of its own, and see if I can make nice examples based on
meta buidbot and SimpleConfig use-cases.

Pierre, I don't forget the dynamic variant, it's just simply farther
away, given what we already have.

Regards,

On 03/23/2016 03:39 PM, Dan Kegel wrote:
>
> fwiw, I'm using a simple static capability system (the OS field in
> https://github.com/buildbot/buildbot/blob/master/master/contrib/SimpleConfig.py
> ), and am thinking of extending it.  So I'm interested in the subject,
> too.
>
> On Mar 23, 2016 5:27 AM, "Georges Racinet" <gracinet at anybox.fr
> <mailto:gracinet at anybox.fr>> wrote:
>
>     On 03/23/2016 01:07 PM, Pierre Tardy wrote:
>>
>>
>>     Le mar. 22 mars 2016 à 22:07, Georges Racinet <gracinet at anybox.fr
>>     <mailto:gracinet at anybox.fr>> a écrit :
>>
>>         Hi there,
>>
>>         After a long time during which we couldn't do much more than
>>         maintaining
>>         our instances, we (Anybox) are considering migrating our
>>         buildbots to Nine.
>>
>>
>>     This is good news!
>>      
>>
>>
>>         Among the many things that I've developed for our needs [1],
>>         there is a
>>         rather complete capability system [3], and hence I must
>>         decide whereas
>>         it's worth porting it or not [4]. I know it's a fairly common
>>         need that many
>>         development shops have been doing in private or almost, but
>>         it's not
>>         obvious to me what is freely available (for Nine) on this topic.
>>
>>         http://trac.buildbot.net/ticket/3120 does not get to a
>>         conclusion, and I
>>         couldn't find much on it in a quick search of recent messages
>>         on this
>>         list. Maybe I missed something else ?
>>
>>
>>     Since that, I figured Workers have properties, which can be used
>>     in NextSlave callback in order to match any capability.
>>     I think it would be much better to have something in the core,
>>     not depending on having the users writing a capability specific
>>     nextSlave/nextBuild callbacks.
>>
>>     Actually, there are two aspect in the capability problem. The
>>     static and the dynamic.
>>
>>     1\ Static problem is the easiest. It looks this is what you
>>     implemented already (and metabuildbot has in a smaller extend):
>>     - One part of your config describe the workers, and their
>>     capabilities
>>     - Another part is describing the builders, and their required
>>     capability.
>>     The system will then automatically configure which worker to
>>     associate to which builders.
>     Yes, that is exactly what I'm doing. It's a bit more actually,
>     since our system also spawns builder variants (applying the same
>     BuildFactory several times for different versions of some
>     capabilities).
>      
>     The full capability description is stored as a dict-valued Worker
>     property, something you may very well call a hack.
>     Then at build time, a custom step extracts the needed properties
>     for that build. This is good enough for us, but it is intrusive :
>     the step must be introduced in the build sequence.
>>
>>     2\ The dynamic problem. Slave are chosen at the start build phase.
>>     - One part of your config describe the workers, and their
>>     capabilities
>>     - Another part is describing the builders, and their required
>>     capability.
>>     - Each build requests can require an additional set of
>>     capabilities, which can restrict more the set of slaves which can
>>     run them.
>>     In order to implement that efficiently, one will have to mess
>>     around inside the buildrequest distributor code, which I can
>>     understand looks scary.
>     I just took a quick tour of that for an unrelated reason
>     (http://trac.buildbot.net/ticket/3498), yes it's a bit convoluted.
>
>>
>>     I am not sure which one you are needing/implementing.
>>     I think it is fine that you only implement 1/ if this is what you
>>     need, but I would like that the design opens the door to
>>     implementing 2 without need to rewrite everything.
>     Interesting. I think I didn't know about nextWorker at the time I
>     started this. What concrete use-case do you have in mind for 2/  ?
>     Also, if the idea is that users don't need to implement 
>     nextWorker/nextBuild, then it means they may implement it for
>     other purposes, and it follows that the dynamic dispatching must
>     play nice with that, isn't it ?
>
>     Regards,
>
>     -- 
>     Georges Racinet
>     Anybox SAS, http://anybox.fr
>     Téléphone: +33 6 51 32 07 27 <tel:%2B33%206%2051%2032%2007%2027>
>     GPG: 0x33AB0A35, sur serveurs publics
>
>
>     _______________________________________________
>     users mailing list
>     users at buildbot.net <mailto:users at buildbot.net>
>     https://lists.buildbot.net/mailman/listinfo/users
>

-- 
Georges Racinet
Anybox SAS, http://anybox.fr
Téléphone: +33 6 51 32 07 27
GPG: 0x33AB0A35, sur serveurs publics

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.buildbot.net/pipermail/users/attachments/20160327/8de84508/attachment.html>