[devel at bb.net] GSOC 2018 & buildbot improvements

Pierre Tardy tardyp at gmail.com
Wed Feb 21 12:13:33 UTC 2018

> > chosen. So we gave up trying.
> If you gave up after just a single failure to apply, I don't really
> understand why giving up. I understand that GSOC team has troubles
> explaining to every single organisation why they were not chosen.

Yes, this was probably more like a threshold, we also found that the admin
work needed was quite big, and wanted to refocus our efforts to development.

> Very often it can be just a big number of *better* applications
> (whatever 'better' means) and the limited number of slots. Just
> curious: did you ever attend the mentor meeting at Google?

Personally, I was never able to attend, but I think we had some mentor
there already.

> > I liked the GSOC mentoring though, and I would be glad to do it again.
> Then you should simply apply.
or co-mentor with another project who do the admin work :)

> > I acknowledge your pain points on Buildbot eight, and agree that nine
> will
> > probably not help that much with the current turnkey visualisation
> plugins.
> > Other people are also frustrated by the nine version of the waterfall,
> > including the webkit people
> https://github.com/buildbot/buildbot/issues/3884
> > I added lengthy explaination in this issue on why it is like that and
> why it
> > is not easy to improve. But help is appreciated.
> >
> > My suggestion for webkit is also valid for macports. What you may need
> is a
> > new UI plugin, which is carefully crafted for your requirements.
> Is this something that can be done in a generic way for other projects
> as well or is this really something that each project should prepare
> from scratch?

The goal is to allow people with reasonable web frontend background to
build their custom dashboard in 2 or 3 days.
Each project does not require that, this is why we try to have some generic

It is much harder to make generic tools than to make specific ones. We had
a lot of issues with console and grid, which was working great with some
people, but for other was not , because of the different SCM, or the
different pipeline setup.
That is why Buildbot is a framework to build specific workflows.

For people with lots of builders, the design trade-off will be different.
If we manage to build a new dashboard for people with lots of builders, I
think this would be even better.
Doing first a specific one is a good first step, then we can show it to
others and  step backward to see how it can be useful to them.

> I understand that probably creating summaries on per-port basis is
> something that should probably be done by us, but I can imagine that
> there should be some solution that would allow anyone to have a
> slightly better view.
I am not sure exactly what you have in mind.
The grid view has builder tag support: https://nine.buildbot.net/#/grid
You could setup your builders with a tag per port, and then use this kind
of filtering in order to have a per port view.
We would need to change the UI for grid view, because obviously the current
UI will be unusable for thousands of tags.

> Just curious: do you have any examples of a super similar plugin that
> goes in this direction?

The examples we have are the console/grid/waterfall plugins.

> > I think this is a very good match for a GSOC project. This would be a
> > project to address your points 3, 4 and 5.
> It would probably help to prepare a rough description together. I know
> what we need to achieve, but I don't know what needs to be done.
I think we first need to have a sketch of the dashboard of how you see data
is displayed. maybe several sketches with options, and then I can help to
translate that with more technical details. This is also good to let some
part of the technical details to the student proposal for easier

> (There's another thing: some potential students asked if it is
> necessary to have access to macOS to be able to apply with our org. My
> answer was that while this is not a strict requirement, it would
> greatly help if they do, in particular it would be nearly impossible
> to keep such students after GSOC is over. But if there are candidates
> who could stick with buildbot org, I don't mind at all.)
Yes, for us it was very hard to keep student around, we had only one who
did contribute from time to time for another 3-4 months.

> > I think this can be done by recruiting a front-end student. Probably this
> > kind is a bit easier to recruit, and the problem are simple enough to
> > enunciate clearly for a student.
> Except that I don't know how front-end students working on Linux/Mac
> would even notice MacPorts to look at project ideas :) :) :)

Well GSoc is supposed to help for that. I believe we can tag the project
front-end or javascript, and this will attract students.

> A new VM would be perfect, but I admit I have no clue what is required
> on macOS to do that.
I think this good be sorted out by the student in the proposal build phase.

> > In any case it is a tough problem that is not easy to resolve as
> basically
> > the goal of the CI is to give developers access to machines, a malicious
> > developer will easily take control of that machine.
> > A potential project for this problem would be to add some extra
> triggering
> > criterias for building a MR, like the approval of at least one trusted
> > member of the team.
> > It looks like a lone a bit short subject for GSoC, I would say.
> It could potentially be listed and then students can decide themselves
> which tasks they want to work on. But I believe it helps a lot if the
> mentor knows how to solve the problems himself/herself and only lacks
> time to do things, else it might be a bad experience in case a student
> is stuck. So if this task gets listed, someone needs to know how to
> implement a solution.

That is why it is a bit harder. It is more like a research task were most
of the work is finding out how to do it, then doing it is relatively
straighforward once you found a solution.

> As far as mentoring is concerned, I know what we need, but I'm not
> familiar with frontend development at all, so I could not help a
> student stuck at technical issues, but could help guiding towards the
> desired goals.
So this is a very good use of co-mentoring I can help with the technical
issue, and you work on the goals.

> Should we draft a project description and publish it on our ideas list
> (and potentially on your website as well in case some students
> interested in helping buildbot development stumble across it)?

Yes, lets start it.
Like said earlier, having a drawing of what would be a good dashboard in
your view is a good start, and then I can add some details on which data
needs to be fetched and how they need to be gathered.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.buildbot.net/pipermail/devel/attachments/20180221/2a810a66/attachment.html>

More information about the devel mailing list