[Buildbot-devel] Port reservation for functional testing

[Georges Racinet]

On May 28, 2012, at 1:59 PM, Dmitry Nezhevenko wrote:

> On Mon, May 28, 2012 at 01:22:57PM +0200, Georges Racinet wrote:
>> Example use-case : launch a web application in build step #n, launch
>> selenium tests in build step #(n+1). Imagine there can be dozens of
>> other services, including several build slaves for different masters
>> on the same host.
>> We also have cases to test in which two applications have to
>> communicate to finish their installation, before integration tests  
>> can
>> be run.
>>
>> It wouldn't be too hard to have step #(n-1) bind to a port, free it
>> and store the number in a property. A SetProperty step launching a
>> small python script could do it. A lock to avoid race conditions in
>> one given slave plus a range parameter to isolate slaves running on
>> the same host should protect against race conditions.
>
> There is also an issue with interrupted build. You need to somehow
> "cleanup" everything and tear down all stuff started during previous
> steps.

Ah yes, you're right, thanls for raising this.
A true tear down is probably very hard to achieve, but avoiding to  
spawn continuously new processes on new ports is doable I think :

- in the service-to-be-tested startup step, store pid in some fixed  
place.
- add a step before the port reservation step to kill any stale  
process from earlier builds, reading that pid file.

Of course we could still have one stale process per builder till the  
end of interruption cause and there's a new build on that slave for it  
but I'd accept that.

Do you think it'll stand ? It'd be really easy to achieve with a  
utility such as start-stop-daemon, which I believe to be Debian  
specific, but may have cross-platform counterparts ; start-stop-daemon  
has a retry option to be quite sure the killing has been done.

Regards