[users at bb.net] Buildbot worker in Jenkins

[Povilas Kanapickas]

Hi Denis,

On 4/11/21 10:34 PM, Denis Ovsienko wrote:
> Hello list.
> 
> I've been tinkering with Buildbot for a couple months, and so far
> managed to get the workers running on a number of architectures and
> Unix-like operating systems. It's been an interesting journey so far.
> 
> There is one specific architecture though, which I can access through
> Jenkins only at the time. The Jenkins instance runs the build via
> Docker, so I've been thinking if it would be possible to run
> buildbot-worker in the container. The reason for this is that the
> master configuration is done programmatically as a product of several
> repositories and several workers, and in Jenkins it takes to create a
> separate build for every repository, and to update the builder specifics
> manually. Also it would be more convenient to manage the status reports
> in one place.
> 
> The Buildbot manual describes Docker latent workers, which assumes that
> the master requests a container from the docker server. However, in this
> case this is not possible.
> 
> That said, the existing "buildbot/buildbot-worker" Docker container
> boils down to running "buildbot-worker start --nodaemon", so Jenkins
> could run that instead of the build commands. This way the Buildbot
> master could have a regular (non-latent) worker permanently configured,
> and Jenkins could start the container with buildbot-worker on a web hook
> from the git hosting service.
> 
> The matter is, the worker container needs to exit after consuming all
> its build request queue. One way to achieve that could be through the
> master's "build_wait_timeout", which would possibly require either a
> new (mostly no-op) latent worker class, or porting the parameter into
> the non-latent worker space. Another way could be through an additional
> optional worker parameter, such as "buildbot-worker start --nodaemon
> --idle-timeout 60", so as soon as the worker has no new work for a
> minute, it exits and the Jenkins build concludes.
> 
> Do you think this would be a useful addition? If somebody implements
> anything along these lines, I would be glad to test the new feature.

In the past I've implemented support for a similar feature where I
Buildbot would suspend the host a worker is running on in order to
reduce noise and electricity costs. I haven't upstreamed all of this
work yet.

It was quite a generic implementation, there's an arbitrary command to
start a machine and an arbitrary command to stop it. So you potentially
could hook this to start/stop Jenkins jobs.

Cheers,
Povilas