[Buildbot] #3657: create single-file distribution of buildbot-worker

[Buildbot trac]

#3657: create single-file distribution of buildbot-worker
------------------------+------------------------
Reporter:  rutsky       |       Owner:  rutsky
    Type:  enhancement  |      Status:  new
Priority:  major        |   Milestone:  undecided
 Version:               |  Resolution:
Keywords:               |
------------------------+------------------------

Comment (by rutsky):

 I made proof-of-concept for building and using single-file buildbot-worker
 using !PyInstaller.

 Here is gist with instructions and scripts to build `buildbot-worker`
 file: https://gist.github.com/rutsky/9f6e5b84050a31b1c4ef10e3a1d86f51

 == Result ==

 Single-file `buildbot-worker` (9.4 MB).

 `buildbot-worker`'s `create-worker`, `start`, `stop`, `restart` commands
 works.

 == How it works? ==

 Here is !PyInstaller's description:
 https://pyinstaller.readthedocs.io/en/stable/operating-mode.html#how-the-
 one-file-program-works
 In short, it unpacks to temporary directory all shared objects on each
 run.
 This means slower start than `normal` buildbot-worker.
 On my system:
 {{{
 # normal
 $ time buildbot-worker > /dev/null

 real    0m0.125s
 user    0m0.092s
 sys     0m0.028s

 # single-file packaged by PyInstaller:
 $ time ./buildbot-worker  > /dev/null

 real    0m0.307s
 user    0m0.252s
 sys     0m0.048s
 }}}

 I don't think this difference is significant in practice.

 == (overcomed) problems building single-file `buildbot-worker` ==

 1. !PyInstaller doesn't work with entry points:
 https://github.com/pyinstaller/pyinstaller/issues/305

 2. !PyInstaller can't package stock `zope.interface`.

    This issue arised several times on !PyInstaller's bugtracker as
 resolved, maybe I missed something.
    To workaround issue with `zope.interface` I convert it from "namespace
 package" to "normal" package by creating `__init__.py` in `zope`
 directory.

 3. Buildbot-worker loads some packages lazily using reflection.

    For example it all command line commands lazily. This can be
 workarounded by explicitly listing all required Python modules.

 == What's next? ==

 What's next?

--
Ticket URL: <http://trac.buildbot.net/ticket/3657#comment:2>
Buildbot <http://buildbot.net/>
Buildbot: build/test automation