[devel at bb.net] worker transition: changes to buildbot/plugins/db.py

[Vladimir Rutsky]

Hi!

On Wed, Mar 30, 2016 at 7:48 PM, Mikhail Sobolev <mss at mawhrin.net> wrote:
> Hi,
>
> (This is mainly for Vladimir Rutsky, but I'd apprecite input from others as
> well.)
>
> Background: I do not expect Buildbot master to be installed for running unit
> test cases, however it seems that now we require that (yes, this was a
> requirement in some other parts of Buildbot, but not for master).

While I don't expect it to work, looks like `trial buildbot/test`
without installation of Buildbot works:

https://travis-ci.org/rutsky/buildbot/jobs/119831925#L6189
https://travis-ci.org/rutsky/buildbot/jobs/119831925#L6488
https://travis-ci.org/rutsky/buildbot/jobs/119831925#L6777

There I install Buildbot first to install all deps, then uninstall it,
and then do `trial buildbot/test/`.

Misha, can you try to run tests in clean virtualenv?

Also works setuptools way of running tests without installation of Buildbot:

$ virtualenv env
$ source env/bin/activate
$ pip install autobahn # Known bug. Other packages are not installed
$ cd master
$ python setup.py test -m buildbot.test.unit.test_plugins
# or
$ python setup.py test -m buildbot.test


> I'm looking at the changes to buildbot/plugins/db.py and I do not really
> understand what exactly it aims to achieve.
>
> I'd appreciate some high level description of the functionality added.

Problem: plugins API uses "slave" teminology which should be fixed (bug 2340).
"slave" is in entry point (namespace): "buildslave", and in classes,
e.g. buildbot.plugins.util.SlaveLock.

What was done: "slave"-stuff was renamed to "worker"-stuff.
Significant constraint: all changes must be backward compatible.

This means:

1. If user's master.cfg uses buildbot.plugins.buildslave and
buildbot.plugins.util.SlaveLock
this must work without modifications in 0.9.0 (and it should print
deprecation warnings).

2. If user's master.cfg uses new buildbot.plugins.worker and
buildbot.plugins.util.WorkerLock
this must work (starting from 0.9.0b7).

3. If user's master.cfg uses third party Buildbot plugin that uses old
entry point (namespace)
"buildslave", this must work with old entry point "buildslave".

Bonus: third party plugin that uses "buildslave" entry point is also
available with new entry point
(namespace) "worker". This allows to use new entry point with old
plugins while they are not upgraded
to use entry point.

4. Third party plugins that uses new entry point "worker" must work as expected.

Here is part of code that handles "buildslave" -> "worker" entry point
(namespace) backward compatibility
handling:
https://github.com/buildbot/buildbot/blob/master/master/buildbot/plugins/db.py#L366

Here is part of code that handles "slave"-named classes backward compatibility:
https://github.com/buildbot/buildbot/blob/master/master/buildbot/plugins/db.py#L424

> What I see is a number of classes added (which do not initialise their parents)

Bunch of your, Misha, classes like _PluginEntry, _NSEntry, _Plugins, _PluginDB
doesn't call parent initializers (which is ok for their current usage).

My _PluginEntryProxy and _DeprecatedPluginEntry doesn't call parent
constructor by design.

> and then a number of [muted] accesses to supposedly private members.

Yes, I use implementation details of some *private* classes of db.py
*inside* db.py,
which is by design.

Misha, I appreciate if you write more concrete examples of code that
you don't like (e.g. lines of code).


Regards,

Vladimir Rutsky