[Buildbot-devel] Some questions/problems running latest build bot source on Windows

[Elliot Murphy]

Hi Brian,
Thanks for committing all those win32 fixes, I've tried them out and they
work well.

|Odd. 'twistd' normally writes a twistd.pid file and then puts itself into
|the
|background. Take a look at the 'twistd.log' file (in the same directory)
|and
|see if shows any obvious errors.
|
|> - Once the buildbot master is running, switching to another console and
|> running buildbot stop doesn't work. It prints out "kill  `cat
|twistd.pid`".
|> I don't see any file named twistd.pid (probably a side effect of the
|> non-forking behavior in buildbot start.
|
|Yup. Although writing the .pid file is independent of the forking behavior.
|
|I know that twistd on windows is a little bit weird, there is a different
|helper module to make it work (named twistw), and a frequent question on
|#twisted is about one of the error messages that happens when you try to
|"run" the helper .py module instead of the actual script. It doesn't sound
|like that's the behavior you're seeing, however.
|It's possible that the way I'm spawning twistd (i.e. with os.system) does
|not
|actually run the right thing.. maybe it's not seeing the right batch file
|or
|something. I could theoretically import the twisted modules and run
|twistd.run() directly, but I suspect that would make windows portability
|worse, not better.

[elliot] 

It looks like the right twistd script is being invoked, so the way that you
are calling twistd doesn't seem to be at fault. Since the process stays
running in the foreground, everything that would normally go to the log file
is printed on the console, and I don't see any errors.

I did some searching/reading on the web, and it appears that twistd does not
have the ability to background the process on Windows because python lacks
os.fork() on Windows. In order to be able to run buildbot masters or slaves
on Windows without requiring that the user stay logged in, the right
solution seems be to enabling the buildbot masters or slaves to run as
Windows Services. Andrew Bennetts posted a pleasantly simple example of how
to make a python script into a service on the Twisted-Python list:
http://twistedmatrix.com/pipermail/twisted-python/2002-February/000820.html 

The buildbot command could probably be adjusted to create the service on
Windows, and call twistd on *nix systems.


|> I've also attempted to run the unit tests, and after making the changes
|I've
|> described already, I get 7 failures, 13 errors, and 11 skips (I've not
|> downloaded the VC repository packages). Would it be useful for me to post
|> some logs from the failed tests?
|
|Yes, that would be great. There's no good reason for them to fail, and it's
|probably because of other unix-centrisms I've used.
[elliot] 

I've attached the unit test logs, running against CVS from this morning.
Here is how I ran the unit tests (executed from the root of the CVS
checkout):

set PYTHONPATH=.
trial -o buildbot.test > trial.log

I'm attaching trial.log and _trial_temp/test.log. I'm happy to re-run or try
patches to help narrow down or solve any of these failures.

|I think you're the first to try a windows buildmaster, although others have
|run buildslaves on w32 before. There are no fundamental reasons why both
|pieces shouldn't be able to run under windows, just lack of
|portability-discipline on my part (and lack of testing). Any patches you
|can
|think of would be warmly welcomed.
|
|That said, I'm a unix guy, not a windows one, and it probably shows in my
|code. It is possible that some of the upcoming features (specifically the
|'try' feature, which will need some kind of secure connection from
|developers
|to the buildmaster, possibly using twisted's "conch" ssh module) may need
|library support that is easier to acquire on linux than on windows. So if
|you're interested purely in having it work, linux may be the more
|mainstream
|choice, but if you're willing to make a few patches, I think we can get it
|to
|work as- or nearly-as- well under windows.
|
|I'm hoping to set up a "meta-buildbot" in the near future, which would run
|the buildbot test suite each time the buildbot's code is changed. If I can
|find a w32 buildslave for this effort, we can make sure that these bugs get
|fixed and stay fixed.
[elliot] 
This sounds great. I saw that Yoz Grahame mentioned contributing a Windows
buildslave for this meta-buildbot; if that one doesn't work out or you need
more than one, I'd be willing to contribute as well.

regards,
-elliot

-------------- next part --------------
A non-text attachment was scrubbed...
Name: test.log.gz
Type: application/octet-stream
Size: 3882 bytes
Desc: not available
URL: <http://buildbot.net/pipermail/devel/attachments/20041108/b4ebc30c/attachment.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: trial.log.gz
Type: application/octet-stream
Size: 2229 bytes
Desc: not available
URL: <http://buildbot.net/pipermail/devel/attachments/20041108/b4ebc30c/attachment-0001.obj>