[Buildbot-devel] GSoC Asynchronous Master/Slave Protocol

[Dustin J. Mitchell]

On Mon, Mar 26, 2012 at 6:11 PM, Tiberiu Paunescu <tpa12 at sfu.ca> wrote:
> I'm interested creating an asynchronus master - slave protocol. I have a rough idea for a schedule, but I think this is something that needs more discussion. I would really appreciate input on this.
>
> 1. Familiarization ~2-3 Weeks
>
> Initially, I need to become familiar with the Buildbot architecture, with an emphasis on master-slave communication. I think some useful deliverables might be sequence diagrams of the communications and other UML diagrams, and improved documentation on the website. This would help me and future developers understand how things work.

There is some detail on this here:
  http://buildbot.net/buildbot/docs/current/developer/master-slave.html
I do like hearing plans for deliverables from this stage - it's
frustrating for a mentor to only hear "I did some reading..", since
it's hard to quantify.

Also, if you're not familiar with Twisted Python, then you may want to review
  http://buildbot.net/buildbot/docs/current/developer/style.html
which talks a lot about how we use Twisted.  It would be great to have
plans for concrete evidence of proficiency with Twisted.  For this
particular project, that may mean implementing some simple network
protocol as a Twisted daemon.

> 2. Design ~4-6 Weeks
>
> I imagine that during this phase, I would work very closely with my mentor. When designing the protocol, it's important to consider the needs of Buildbot, and any work already done on this. I like zeromq, and I have a lot of experience programming in C. Deliverables would be the design documents created.
>
> 3. Implementation ~4-6 Weeks
>
> Once my mentor and I have agreed upon a design, and I've documented my intention for the protocol, I can begin implementation. I think creating a design that satisfies the needs of Buildbot and is well documented is more important than actually implementing it. If I don't complete the implementation, I can always work on it after the GSoC based on the design.
>
> 4. Testing ~2 Weeks
>
> Any remaining time would be spent on testing.

These three steps are probably best done simultaneously, actually.  We
generally like to see new functionality implemented as a sequence of
small, easily reviewed patches, each complete with tests and
documentation.  If necessary, that sequence of patches can be kept on
a non-master branch, and only merged when it's "ready", but this
avoids the problem of you working for weeks to churn out thousands of
lines of code, then sitting on your hands while one of the developers
tries to read through it -- with the right granularity of patches,
review and work on the next part can proceed in parallel.  Hitting
this balance is one of the harder parts of doing signficant work in
OSS, as you probably know from last year.

> I'm currently on vacation, and I don't have access to a computer that I can configure for building. However, I would like to set up a build machine for Worldforge's Ember client. Looking at the documentation for Buildbot it seems that build scripts can only be configured in python. Ideally, I think that the build process should allow for configuration in any language.
>
> I'm really interested in distributed systems, and I'd like to work with Buildbot this summer and in the future.

That sounds great -- a reason to continue using the application
indicates to us that you're likely to keep *contributing* to it, as
well, which is one of the program goals.

Dustin