[users at bb.net] Buildbot-worker 0.9.3 fails to start on Windows

Tue Jan 17 13:57:11 UTC 2017

Hi Grzegors,

I agree with your statement that we should support a default value to
cpu_count, and also we should document that on windows NUMBER_OF_PROCESSOR
variable should be set.

I would love to see a pull request about that

Regards
Pierre

Le mar. 17 janv. 2017 à 09:59, Grzegorz Antoniak <ga at anadoxin.org> a écrit :

> Hello Pierre and Neil,
>
> Thank you for both answers. Especially the `appveyor.yml` file was
> helpful. I've managed to get buildbot-worker v0.9.3 running on
> Python v2.7.6.
>
> I'll add some details in case someone will stumble upon this thread in
> the future:
>
> The patch I wrote about in my original mail turned out to be necessary
> only when the environment is missing the NUMBER_OF_PROCESSORS
> environment variable. It appears that this variable is used by
> <python2.7>/lib/multiprocessing/__init__.py, which uses it inside the
> implementation of the `cpu_count` function. If this variable is missing,
> the function throws NotImplementedError instead of returning a default
> of 1 cpu, and `buildbot-worker` doesn't catch this exception.
>
> While I understand that a missing NUMBER_OF_PROCESSORS variable is a
> problem on my side, I think buildbot-worker also might have a bug in
> this case, because Python documentation states that `cpu_count()`
> function is allowed to throw an exception. I think it might be a good
> idea to catch it in the code, and set worker's CPU count to 1, if it
> will occur.
>
> Anyway, setting the correct NUMBER_OF_PROCESSORS variable fixed the
> problem for me.
>
> Thank you for your help,
> Grzegorz Antoniak
>
> On Mon, Jan 16, 2017 at 09:28:20AM -0600, Neil Gilmore wrote:
> > Hi everyone,
> >
> > Our Windows workers are generally running on 2.7.12, with a few older
> back
> > to 2.7.6 (I think, I'm too lazy to look). We start them using ssh under
> > Cygwin, if that matters to you. However, the few running a release
> version
> > are running 0.9.1. The others are running 0.9.9rc2(?) or so. I don't
> recall
> > any significant patches. We aren't running any masters on Windows.
> >
> > Neil Gilmore
> > grammatech.com
> >
> > On 1/16/2017 8:25 AM, Pierre Tardy wrote:
> > >Hi,
> > >
> > >Buildbot at the moment only works with python 2.
> > >We are working at the python3 port but it is not quite ready (I'm hoping
> > >we can have worker support for 0.9.4 thanks to the great work from
> Craig)
> > >
> > >As per the Windows support, we are supporting windows by running unit
> test
> > >suite and integration suite on appveyor.
> > >However, according to our statistics, we have no user accepting to send
> > >info to the statistic server and running windows. :-(
> > >So I cannot say "it works, we have people running it". We are however
> > >interested in having windows users, and are willing to help as much as
> we
> > >can.
> > >
> > >You can see on the appveyor.yml how we install pywin32 and other
> buildbot
> > >dependencies:
> > >https://github.com/buildbot/buildbot/blob/master/appveyor.yml#L13
> > >
> > >Let us know.
> > >Pierre
> > >
> > >
> > >Le lun. 16 janv. 2017 à 12:15, Grzegorz Antoniak <ga at anadoxin.org
> > ><mailto:ga at anadoxin.org>> a écrit :
> > >
> > >    Hello,
> > >
> > >    I would like to know what is the preferred version of Python for
> > >    running
> > >    buildbot-worker on Windows?
> > >
> > >    Currently I'm trying 0.9.3 on Python 3.6, but it fails with an
> error:
> > >
> > >        File "C:\python36\lib\site-packages\twisted\spread\pb.py",
> > >    line 1076 [...]
> > >            m.update(password)
> > >        builtins.TypeError: Unicode-objects must be encoded before
> hashing
> > >
> > >    I've also tried 0.9.2 and 0.9.1, same error.
> > >
> > >    I've tried to install 0.9.3 on Python 2.7, but it failed with this
> > >    error:
> > >
> > >        File "c:\python27\lib\site-packages\buildbot_worker\base.py",
> > >    line 315[...]
> > >            self.numcpus = multiprocessing.cpu_count()
> > >        File "c:\python27\lib\multiprocessing\__init__.py", line 136,
> > >    in cpu_count
> > >            raise NotImplementedError('cannot determine number of cpus')
> > >        exceptions.NotImplementedError: cannot determine number of cpus
> > >
> > >    I've checked the documentation for `multiprocessing.cpu_count`, and
> it
> > >    says that this function can throw NotImplementedError, as it
> > >    happens on
> > >    my machine. I've added a simple patch to catch this exception in
> > >    `base.py` around line 315, the error was gone and the worker started
> > >    successfully, but now I'm having some problems installing `pywin32`
> > >    (which are not related to this mailing list) so the worker is
> > >    still not
> > >    fully operational.
> > >
> > >    So I'm wondering, what is the Python setup of the developers? Which
> > >    version of Python is known to work on Windows without any patches?
> > >
> > >    Regards,
> > >    Grzegorz Antoniak
> _______________________________________________
> users mailing list
> users at buildbot.net
> https://lists.buildbot.net/mailman/listinfo/users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.buildbot.net/pipermail/users/attachments/20170117/13846d96/attachment.html>