[devel at bb.net] Buildbot dependencies

[Denis Ovsienko]

Hello list.

Thank you for maintaining this project, which enables many open source
projects to run CI without vendor lock-in.  In particular, you might be
interested to know that it powers a significant portion of the
multiple-OS and multiple-architecture CI for the software projects of
the tcpdump group at https://ci.tcpdump.org/

Based on this experience I would like to provide some feedback about
the pip packages of Buildbot.

First of all, the web site at buildbot.net used to say: "Buildbot is
dropping support for Python 2.7 for the master.  Users are encouraged to
migrate to Python 3 as soon as possible."

However, this notice seems to be out of date.  About a year ago I tried
to setup a Buildbot worker on a Python 2.7 system via pip, and that
turned out to be impossible even with much older package versions.  The
matter was, one of the Buildbot worker key dependencies (twisted?) is
not compatible with Python 2.7 anyway, so there is no support for
Python 2.7 in the first place.  I guess the same likely applies to the
master component.

This peculiarity does not make much practical difference in my use case
because all the current workers have Python 3, but perhaps you would
like to rephrase the notice.

Another thing is, and it does have unwanted practical consequences, that
the worker package starting with version 3.5.0 started to depend on the
"cryptography" package, which is not just Python code, but a mix of
Python and Rust, which requires a Rust compiler.

The problem with the latter is that not every OS has a Rust compiler.
Common and well-supported cases, such as Linux/AMD64, typically have
Rust as a distribution package, so it can be installed with a single
command in a couple minutes.  For other operating systems and/or
architectures Rust is available only as a port, i.e. the Rust compiler
itself must be compiled from its source first.  This is sometimes
difficult.

As one example, on NetBSD/AArch64 it takes 3 days to compile a number
of packages (including Clang and GCC) from scratch.  The same system
spent about 7 days trying to compile Rust before I ran out of patience.
That's even if the Rust port is readily available.  The additional Rust
luggage is very unpracticable, especially for the functionality that is
unused in the first place.

The matter is, my setup very well handles all cryptography completely
outside of the Python space (both at the worker end and the master
end), so until version 3.5.0 it was completely fine not to install the
Python/Rust "cryptography".  Unfortunately, the additional new
dependency in version 3.5.0 has broken the modus operandi.  Could
anyone explain the reason for the additional dependency?  Would it be
OK to remove the new dependency and to make the worker package like it
used to be before?

-- 
    Denis Ovsienko