[Buildbot-devel] question and thoughts concerning parallel building and locks

[Alexander Lorenz]

hi,

i have six similarly configured buildslaves and six builds which i want 
to run on them.


i don't want to use static distribution by "hardwiring" one slave to 
each builder, but would rather have dynamic distribution like this:

a builder starts building on the first free buildslave on its list it 
finds, if none is free, it waits until one becomes free.

i don't know if that's possible with the 'slavenames' and 'locks' lists 
which you can assign to builders.
and to be honest, i'm a little confused about the deeper mechanisms of 
the lock implementation ...


first step: i assign all six slaves to each builder.

that alone does no good: the builders do not 'notice' (or don't care) if 
another builder is already building on a particular slave and do not 
pick another slave even if they have multiple slaves assigned to them: 
in  a test all builds ended up on the last slave in the 'slaves' list.
they _do_ have a notion of 'busy' for their _own_ build though: if a 
builder runs a build again, before a former run of the same 
builder/build is completed, it picks another slave from it's slavelist.


second steps: adding locks.

i can't seem to find a clever way via the locks in bb as i understand them.
assigning six locks to all builders is no use: the first builder grabs 
all six and the others wait, same as with just one lock ...

so am i overlooking something here?

and does it sound like a good idea to extend the "free slave awareness" 
of a builder beyond it's own build, considering all running builders?

still, even if they had this awareness, without 'proper' locks the above 
described 'wait' condition would still be unfulfilled: if all slaves 
were busy, the next builder would just pick a slave nevertheless instead 
of waiting ...


a more general way of designing the behaviour i want would be the 
ability to join identically configured buildslaves into 'slavegroups' 
and assigning these groups to builders.
these groups would manage buiildslave assignment internally via a 
semaphore as big as their number instead of a lock.

oops... i think i'm drifting too far off ... ;)



hope i don't sound too confusing ....

lx