[Buildbot-devel] How to handle firing N jobs into a compute grid and reporting status?

[William Deegan]

Greetings,

I have a gridengine compute grid I'm using to run a series of regression 
steps.
Currently I fire them off via a script and then add N logfiles to the 
build step to monitor their progress and have access to their logs.

This yields one big box with lots of links in the waterfall.

Ideally I'd end up with N boxes (1 for each test) with links to each 
tests logfile(s). Which is what we had prior to switching to use the grid.

I see that Trigger steps can be set to not wait for the trigger to 
return, and actually this is how some of the tests were launched before, 
with many buildslaves able to pick up the tests yielding some job 
distribution.

Is there anyway to have a ShellCommand not wait for completion and still 
process the logfiles?

Barring that I'm thinking that adding N build steps after the test 
launch step, and somehow getting each step to pick up the job completion 
of the last job completed.

So if I launch 3 jobs, there would be 4 steps:

1. launch 3 jobs
2. watch for completion of any of 3 jobs
3. watch for completion of any of 2 jobs
4. watch for completion of the remaining job.
Is there a way to reduce the list of jobs passed from 2->3->4?

Any other suggestions on how to handle this?
The main constraint is I want to see all reporting for a given build in 
a single column, each tests results in a single table cell, and ideally 
be able to see all the tests logs in progress and not just when their 
complete.

Thanks,
Bill