[users at bb.net] Gitlab, MRs, and try builds, oh my

[Dan Kegel]

Continuing with my go at upgrading to current buildbot.
Very happy that clicking on a build shows a clickable
link to the gitlab commit being built!

But merge requests are not yet happy.
In days of yore, my old gitlab trybuild gateway read source_branch
from the merge request,
and used that to decide which branch to run the try build against.
Experimenting (see below), it appears the spiffy new gitlab
integration doesn't do that.

Looking at both the current code
https://github.com/buildbot/buildbot/blame/master/master/buildbot/www/hooks/gitlab.py
and the commit that added gitlab trybuild support (
https://github.com/buildbot/buildbot/commit/080603ea4e6845931a04d021f985079c518f06c7
),
I don't see that happening, confirming that the merge request hook
doesn't do what I think I want.

I guess this is one of those situations where the proper response is
"patches gratefully accepted" :-) so I'll have a look.

There is one further wrinkle: my old buildbot instance used naming conventions
to map project + branch to trybuilders; my old gateway used this awful
code fragment:

    builder_regex = '^%s-(win|osx|ubu)[^-]*-%s-trybuilder' % (project, ubranch)
    try:
        builders_str = subprocess.check_output(
           "buildbot try -c pb --username=%s --passwd=%s -m %s
--get-builder-names | egrep -w '%s'" % (
        try_user, try_pass, try_server, builder_regex), shell=True)

so I would want to find an elegant way to express things like that, too.
(Presumably it's easy to get a list of builder names from inside the
gitlab hook.)
- Dan

p.s. Here's my config, and the logs from accepting push and merge
request notifications:

c['schedulers'].append(schedulers.Try_Userpass(
    name='try',
    builderNames=['runtests'],
    port=5555,
    userpass=[('xyzzy','plover')]))

c['www'] = dict(port=8010,
    auth=util.GitLabAuth("https://gitlab.example.com", "foo", "bar"),
    change_hook_dialects=dict(
         gitlab={
                     'secret': 'baz',
         },
     ),
     plugins=dict(waterfall_view={}, console_view={}, grid_view={}))

On the gitlab side, in the project in question's integrations tab, I
have a web hook
   http://buildhost.example.com:8010/change_hook/gitlab
set to send push and merge request notifications.

Users can authenticate with gitlab just fine, and when they
push commits to gitlab, builds occur promptly on buildbot;
twistd.log shows

2018-05-14 07:52:34-0700 [_GenericHTTPChannelProtocol,21,10.10.169.29]
New revision: 2adedfc2
2018-05-14 07:52:34-0700 [_GenericHTTPChannelProtocol,21,10.10.169.29]
Received 1 changes from push gitlab event
2018-05-14 07:52:34-0700 [_GenericHTTPChannelProtocol,21,10.10.169.29]
WARNING: change source is using deprecated self.master.addChange
method; this method will disappear in Buildbot-1.0.0
2018-05-14 07:52:34-0700 [-] added change with revision
2adedfc2b8d0ea4f0568795fa132d94d2de4f345 to database
2018-05-14 07:52:34-0700 [-] injected change
Change(revision='2adedfc2b8d0ea4f0568795fa132d94d2de4f345', who='Dan
Kegel <dank at kegel.com>', branch='master', comments='Another silly
change.\n', when=1526309542, category='push', project='',
repository='git at gitlab.oblong.com:dank/hello.git', codebase='')
2018-05-14 07:52:34-0700 [-] added buildset 8 to database

However, try builds aren't working just yet.  When I push to a new
branch to prepare a MR, twistd.log shows:

2018-05-14 07:54:11-0700 [_GenericHTTPChannelProtocol,29,10.10.169.29]
New revision: 92268bc7
2018-05-14 07:54:11-0700 [_GenericHTTPChannelProtocol,29,10.10.169.29]
Received 1 changes from push gitlab event
2018-05-14 07:54:11-0700 [_GenericHTTPChannelProtocol,29,10.10.169.29]
WARNING: change source is using deprecated self.master.addChange
method; this method will disappear in Buildbot-1.0.0
2018-05-14 07:54:11-0700 [-] added change with revision
92268bc781b24f0a61b907da062950e9e5252a69 to database
2018-05-14 07:54:11-0700 [-] injected change
Change(revision='92268bc781b24f0a61b907da062950e9e5252a69', who='Dan
Kegel <dank at kegel.com>', branch='foob2', comments='Remove the dummy
line again\n', when=1526309644, category='push', project='',
repository='git at gitlab.oblong.com:dank/hello.git', codebase='')

and no build is triggered; that's probably fine.
But then when I click 'submit merge request', I expect a buildset to
be added, but none is:

2018-05-14 07:54:57-0700 [_GenericHTTPChannelProtocol,30,10.10.169.29]
Received 1 changes from merge_request gitlab event
2018-05-14 07:54:57-0700 [_GenericHTTPChannelProtocol,30,10.10.169.29]
WARNING: change source is using deprecated self.master.addChange
method; this method will disappear in Buildbot-1.0.0
2018-05-14 07:54:57-0700 [-] added change with revision
92268bc781b24f0a61b907da062950e9e5252a69 to database
2018-05-14 07:54:57-0700 [-] injected change
Change(revision='92268bc781b24f0a61b907da062950e9e5252a69', who='Dan
Kegel <dank at kegel.com>', branch='foob2', comments="MR#5: Remove the
dummy line again\n\nLet's see what twistd.log says", when=1526309644,
category='merge_request', project='',
repository='https://gitlab.oblong.com/dank/hello.git', codebase='')

Gitlab merge request notifications contain both a source_branch (in
this case, foob2)
and a target_branch (in this case, master)... but the latter isn't
shown in the log

Well, ok, time to look at the code :-)