[users at bb.net] Buildbot-worker 0.9.3 fails to start on Windows

[Grzegorz Antoniak]

Hello Pierre and Neil,

Thank you for both answers. Especially the `appveyor.yml` file was
helpful. I've managed to get buildbot-worker v0.9.3 running on
Python v2.7.6.

I'll add some details in case someone will stumble upon this thread in
the future:

The patch I wrote about in my original mail turned out to be necessary
only when the environment is missing the NUMBER_OF_PROCESSORS
environment variable. It appears that this variable is used by
<python2.7>/lib/multiprocessing/__init__.py, which uses it inside the
implementation of the `cpu_count` function. If this variable is missing,
the function throws NotImplementedError instead of returning a default
of 1 cpu, and `buildbot-worker` doesn't catch this exception.

While I understand that a missing NUMBER_OF_PROCESSORS variable is a
problem on my side, I think buildbot-worker also might have a bug in
this case, because Python documentation states that `cpu_count()`
function is allowed to throw an exception. I think it might be a good
idea to catch it in the code, and set worker's CPU count to 1, if it
will occur.

Anyway, setting the correct NUMBER_OF_PROCESSORS variable fixed the
problem for me.

Thank you for your help,
Grzegorz Antoniak

On Mon, Jan 16, 2017 at 09:28:20AM -0600, Neil Gilmore wrote:
> Hi everyone,
>
> Our Windows workers are generally running on 2.7.12, with a few older back
> to 2.7.6 (I think, I'm too lazy to look). We start them using ssh under
> Cygwin, if that matters to you. However, the few running a release version
> are running 0.9.1. The others are running 0.9.9rc2(?) or so. I don't recall
> any significant patches. We aren't running any masters on Windows.
>
> Neil Gilmore
> grammatech.com
>
> On 1/16/2017 8:25 AM, Pierre Tardy wrote:
> >Hi,
> >
> >Buildbot at the moment only works with python 2.
> >We are working at the python3 port but it is not quite ready (I'm hoping
> >we can have worker support for 0.9.4 thanks to the great work from Craig)
> >
> >As per the Windows support, we are supporting windows by running unit test
> >suite and integration suite on appveyor.
> >However, according to our statistics, we have no user accepting to send
> >info to the statistic server and running windows. :-(
> >So I cannot say "it works, we have people running it". We are however
> >interested in having windows users, and are willing to help as much as we
> >can.
> >
> >You can see on the appveyor.yml how we install pywin32 and other buildbot
> >dependencies:
> >https://github.com/buildbot/buildbot/blob/master/appveyor.yml#L13
> >
> >Let us know.
> >Pierre
> >
> >
> >Le lun. 16 janv. 2017 à 12:15, Grzegorz Antoniak <ga at anadoxin.org
> ><mailto:ga at anadoxin.org>> a écrit :
> >
> >    Hello,
> >
> >    I would like to know what is the preferred version of Python for
> >    running
> >    buildbot-worker on Windows?
> >
> >    Currently I'm trying 0.9.3 on Python 3.6, but it fails with an error:
> >
> >        File "C:\python36\lib\site-packages\twisted\spread\pb.py",
> >    line 1076 [...]
> >            m.update(password)
> >        builtins.TypeError: Unicode-objects must be encoded before hashing
> >
> >    I've also tried 0.9.2 and 0.9.1, same error.
> >
> >    I've tried to install 0.9.3 on Python 2.7, but it failed with this
> >    error:
> >
> >        File "c:\python27\lib\site-packages\buildbot_worker\base.py",
> >    line 315[...]
> >            self.numcpus = multiprocessing.cpu_count()
> >        File "c:\python27\lib\multiprocessing\__init__.py", line 136,
> >    in cpu_count
> >            raise NotImplementedError('cannot determine number of cpus')
> >        exceptions.NotImplementedError: cannot determine number of cpus
> >
> >    I've checked the documentation for `multiprocessing.cpu_count`, and it
> >    says that this function can throw NotImplementedError, as it
> >    happens on
> >    my machine. I've added a simple patch to catch this exception in
> >    `base.py` around line 315, the error was gone and the worker started
> >    successfully, but now I'm having some problems installing `pywin32`
> >    (which are not related to this mailing list) so the worker is
> >    still not
> >    fully operational.
> >
> >    So I'm wondering, what is the Python setup of the developers? Which
> >    version of Python is known to work on Windows without any patches?
> >
> >    Regards,
> >    Grzegorz Antoniak