[Buildbot-devel] suggestions on how to deal with batch queues and mpi

[Chris Kees]

Hi,

I have some buildslaves for which any tests have to be run through a batch
system, and they must be run via an mpi launcher inside.  For example, if I
was logged in I would do something like

login-node%qsub -I
(wait for interactive session to start on compute node of cluster)
compute-node-0% cd my_tests; aprun python -c "import nose; nose.run()"

I'm wondering if anybody has any cool ways of dealing with this.
Unfortunately one of the systems will only take i/o from a terminal when in
interactive mode (the "-I" switch), so I haven't been able to wrap
something like subprocess.Popen around it.  I was thinking I would write a
script that would inject the commands I want to run into a standard batch
script for a given system and submit it to the queue, but I'm not sure how
to get my script to wait until the job runs and then dump the stdout/stderr
back to the Shell processes.

Thanks,
Chris
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://buildbot.net/pipermail/devel/attachments/20120306/7e74c87c/attachment.html>