[Buildbot-devel] Some questions/problems running latest buildbot source on Windows

[Brian Warner]

> Buildbot looks really cool, so please excuse my unfamiliarity while I'm
> trying to get acquainted with it and reporting a few problems.

Howdy.. Thanks for giving Buildbot a try!

> 1 - buildbot/scripts/runner.py has some platform specific copy commands in
> the createMaster() function that won't work on Windows. Replacing the
> os.system() calls to "cp" with calls to shutil.copy() fixed this.

Ok, committed to CVS.

> 2 - The createMaster() function attempts to install "sample.mk" into the new
> master directory, but fails because it could not find the source file. This
> happens because setup.py is installing the sample.cfg file into the
> site-packages\buildbot directory, but not sample.mk. Changing setup.py to
> also install sample.mk fixed this problem.

Committed.

> - It would be nice if setup.py added "contrib/windows/buildbot.bat" to the
> list of scripts to be installed when sys.platform == 'win32'
> 
> - contrib./windows/buildbot.bat should have an @ sign prefixing the python
> command (this is similar to echo off)
> 
> - buildbot/test/test_maildir.py MailDirTest.deliverMail() has a call to cp
> via os.system that should be replaced with shutil.copy()

Done. Let me know if I got it right.. I'm not a windows guy, so I'll have to
take your word for it.

> - Running "buildbot master", then "buildbot start" gives me a running
> script, and I can access the web server in that script, but the process is
> not put into the background.

Odd. 'twistd' normally writes a twistd.pid file and then puts itself into the
background. Take a look at the 'twistd.log' file (in the same directory) and
see if shows any obvious errors.

> - Once the buildbot master is running, switching to another console and
> running buildbot stop doesn't work. It prints out "kill  `cat twistd.pid`".
> I don't see any file named twistd.pid (probably a side effect of the
> non-forking behavior in buildbot start.

Yup. Although writing the .pid file is independent of the forking behavior.

I know that twistd on windows is a little bit weird, there is a different
helper module to make it work (named twistw), and a frequent question on
#twisted is about one of the error messages that happens when you try to
"run" the helper .py module instead of the actual script. It doesn't sound
like that's the behavior you're seeing, however.

It's possible that the way I'm spawning twistd (i.e. with os.system) does not
actually run the right thing.. maybe it's not seeing the right batch file or
something. I could theoretically import the twisted modules and run
twistd.run() directly, but I suspect that would make windows portability
worse, not better.

> I've also attempted to run the unit tests, and after making the changes I've
> described already, I get 7 failures, 13 errors, and 11 skips (I've not
> downloaded the VC repository packages). Would it be useful for me to post
> some logs from the failed tests?

Yes, that would be great. There's no good reason for them to fail, and it's
probably because of other unix-centrisms I've used.

> Most of the things I've found so far seem really trivial to fix. If you
> don't intend to support running buildbot masters on Windows, let me know and
> I'll focus instead on running the master from a Linux box and switch over to
> trying to get a slave working on Windows.

I think you're the first to try a windows buildmaster, although others have
run buildslaves on w32 before. There are no fundamental reasons why both
pieces shouldn't be able to run under windows, just lack of
portability-discipline on my part (and lack of testing). Any patches you can
think of would be warmly welcomed.

That said, I'm a unix guy, not a windows one, and it probably shows in my
code. It is possible that some of the upcoming features (specifically the
'try' feature, which will need some kind of secure connection from developers
to the buildmaster, possibly using twisted's "conch" ssh module) may need
library support that is easier to acquire on linux than on windows. So if
you're interested purely in having it work, linux may be the more mainstream
choice, but if you're willing to make a few patches, I think we can get it to
work as- or nearly-as- well under windows.

I'm hoping to set up a "meta-buildbot" in the near future, which would run
the buildbot test suite each time the buildbot's code is changed. If I can
find a w32 buildslave for this effort, we can make sure that these bugs get
fixed and stay fixed.

> Many thanks for such an interesting tool.

Here's hoping you find it useful too! :)

cheers,
 -Brian