[Buildbot-devel] Setting a buildslave's active status

[Brian Warner]

> I have a need for one particular buildslave to only trigger it at night. The
> buildslave is running on a machine 24h a day, but the owner only wants it
> run builds when he's out of the office, so say between 7PM and 7AM. 

Seems quite reasonable.

> Is there currently a possibility to make this kind of setup? I didn't find
> it in the documentation or code.

Nope, not yet.

> If this isn't possible right now, I'd like to add it. Looking over the code,
> I'll probably add an 'active' flag to the SlaveBuilder and maybe add a
> callback for checking the current time. In Builder.maybeStartBuild a test
> can be added for this active flag ( in the line 'if sb.state == IDLE:'). 

The problem with an 'active' flag is in getting notification when it gets
set. The general approach in the buildbot is to call a function with a name
like 'waitUntilReady', which returns a Deferred[1], and put off doing
whatever it is you need the thing to be Ready for until that deferred fires.
If the thing is ready right away, the deferred fires right away. This handles
both the immediate case and the wait-for-it case. (using a simple boolean
'active' flag usually results in some sort of polling, and if the interval is
too short then you get lots of unnecessary activity, and if it is too long
then you don't take full advantage of the buildslave's availability).

It might make sense to have this driven from the buildslave side, basically
just not connecting until 7pm and then forcibly disconnecting at 7am. You
could drive this with a cronjob that did 'buildbot start' and 'buildbot stop'
at the appropriate times. The disadvantage here is that the buildslave might
be in the middle of a build when it shuts down.

Another approach might be to have the buildslave send a "presence" status at
the buildmaster, kind of like instant messaging clients do. If the buildslave
wants to stay connected but not accept any new builds, it sets an "away"
status. Later it can set an "accepting work" status, which would prompt the
buildmaster to give it a pending build. Buildslave shutdown could be made
nicer by having it set "away" first, then waiting until any current builds
had been completed before actually terminating. 'buildbot stop' could be
extended to have a --soon or --graceful option that would ask the buildslave
to do this sort of thing.

(of course you'd have to extend the buildslave.tac file to provide a way to
configure this sort of thing. It would probably be a good idea to add a
little web server on the buildslave side at the same time to allow the
build*slave* admin to check the current status, take it offline, etc. This
needs to be done in a way that doesn't complicate the normal always-online
setup, since buildslaves are meant to be installed and configured by
volunteers who shouldn't have to learn a lot to be able to contribute CPU
time).


And actually this might make another long-requested feature[1] easier to
implement: scheduling config changes to avoid interrupting current builds. At
the moment, when you change the config of a Builder, any builds currently in
progress for that builder are interrupted (and lost), so the Builder can be
restarted with the new configuration. Worse yet, I think I coded it to
pretend the buildslave has detached and then reattached, to make sure there
wasn't any state left lying around from the previous config, which means that
reconfiguring one builder might kill off *all* builds running on the same
buildslave. (I'm not sure about that last behavior.. it's been a while since
I looked closely at that code).

The result is that you have to watch the buildmaster carefully and wait until
it is idle before changing its config, if you want to avoid abandoning any
current builds. For a moderately busy buildmaster, it may never actually be
completely idle. It would be awfully nice if you could tell it to stop
scheduling new builds until everything was idle at the same time, and *then*
reload the config file. Having a way to let the buildslaves set an "away"
status (or more specifically, having the buildmaster/schedulers pay attention
to such a status) would give us a place to combine that with a
buildmaster-controlled "pause" state, which would make this
synchronized-reconfig thing a lot easier to implement.


So I'd suggest starting with that. Each buildslave should have an
availability state like you suggested, either AVAILABLE or
NOT_ACCEPTING_NEW_BUILDS, the Builder should aggregate the status of all the
buildslaves that it is allowed to use, and the build-scheduling logic in the
Builder should treat a NOT_ACCEPTING_NEW_BUILDS status much like the
buildslave being completely offline. The UI for setting this state is an open
question, whether to configure this from the buildmaster side (probably by
replacing the c['slaves'] botname/password tuple with a series of BuildSlave
instances, each of which takes some required parameters like name= and
passwd= but also some optional ones like onBreakDuring=(starttime,stoptime)),
or from the buildslave side (with more options in the .tac file). It's also
an open question about how to let the *buildslave* admin pause their
buildslave.. command line tool? local web page?


anyways, good idea.. let's figure out how to implement this.

 -Brian





[1]: SF#1251484: https://sourceforge.net/tracker/index.php?func=detail&aid=1251484&group_id=73177&atid=537004