[Buildbot-devel] Daily builds

[Brian Warner]

> This new scheduler sounds interesting. I don't think we're really using
> BB how it was intended, i.e. one target per slave with incremental steps
> to create an end target.

I don't quite understand what you're doing here, but yeah, the intention was
to have each Builder run a series of Steps, in sequence, and for each Builder
to run more or less independently. "Dependencies" (when they're complete) and
"Interlocks" (as exist in 0.6.6) are meant to express the idea that some
builds just aren't worth doing if some other build has failed completely..
i.e., for Twisted, we'd say that the Builder which creates .deb files doesn't
run unless the tests pass on all platforms. This way we have .deb files which
might be out of date, but which we know will work.

The whole merging-code-from-multiple-sources stuff we've been talking about
(sub-projects and libraries and all) will require some more thought,
especially w.r.t. how to represent the status of it all.

> Reading Michaels and a couple of other mails again it sounds like there
> is possible need for more flexible conditional stepping system. Perhaps
> you could do something like create a "status" object when a build is
> created and then this is passed to each of the steps (perhaps a pickled
> object is passed around?).

The idea is to use the buildbot.process.base.Build object for this purpose.
BuildSteps are free to add whatever attributes they like to their parent
Build, and to examine those attributes before deciding what command should be
run. This is already used by, e.g., the buildbot.process.step_twisted.HLint
step (which runs a "lint"-style check on the .xhtml documentation files): the
step only runs the hlint tool on the changed .xhtml files, which it obtains
by asking the Build for a list of all files mentioned in all Changes, and
then just filters out the ones where file.endswith(".xhtml") is True.
Likewise there is "only run unit tests for the .py files that have changed"
logic in the Trial step.

If the actual sequence of Steps needs to be changed, the Build is again the
place to do this. You'd just override the getNextStep() method to sometimes
insert a new BuildStep. There's an RFE pending to make status-reporting work
with this, but that should be a quick change, once I get through this
Scheduler stuff.

> Other features that might be nice would be the ability to email different
> people depending on the step that failed.

Yeah, this could be implemented by a StatusTarget which just walks through
the BuildStepStatuses and examines their individual results:

 def buildFinished(self, builderName, build, results):
     for s in build.getSteps():
         result, text = s.getResults()
         if result == FAILURE:
             print "step %s failed" % s.getName()

It seemed to me that this is the sort of thing that's best left up to a
custom StatusTarget class. That way the user can configure everything else it
will need (a table mapping step name to a list of email addresses, for
example) in a sensible way, rather than stuffing email addresses into the
Build or something.

> It would also be nice if the status object contained the changes that
> sparked this build - as you might look at paths or comments that were made
> for hints what went wrong.

Yup, again the StatusTarget has access to these through the BuildStatus
object that it receives:

 def buildFinished(self, builderName, build, results):
     files = [f for c in build.getChanges() for f in c.files]
     comments = [c.comments for c in build.getChanges()]

> If you did put in support for passing objects around like this slaves
> might dump out "reports" which could be linked to from the steps. E.g. a
> step might produce some stats in HTML and then post this back to the
> master. The master could then cache this with the logs and link it into
> the waterfall. The logs would still be used to track the build process,
> but the reports would give more palatable information about what is
> going on.

Hmm, interesting. At the moment we have almost all of the HTML isolated into
the buildbot.status.html module: all of the status-reporting interfaces deal
in terms of SUCCESS/WARNING/FAILURE, or short text strings, or logfiles. This
isolation feels like a good idea, so my inclination would be to find a way to
enhance IBuildStepStatus to provide more information than results/text/color,
and then teach Waterfall() or other HTML-oriented status targets to render
this new information into HTML. That way you've got a chance of exposing this
same information through other, non-HTML channels (like IRC or email).

I very much agree that logfiles are a kind of worst-case status delivery
method. Summary logfiles are a tiny bit better, but they're still just a big
string, and there are far more meaningful ways to express certain kinds of
data. Note that when the Step creates a LogFile, it can instead create an
HTMLLogFile object, which is defined to hold one big HTML string instead of a
list of text chunks from stdout/stderr/header channels. The Waterfall page
already knows that when you render an HTMLLogFile you should just dump it
onto the screen. The only thing that uses this in the standard distribution
is the error-handling code that report an HTMLized stack trace when there's
an exception inside the BuildBot somewhere (which hopefully isn't happening
too frequently these days), but you could easily create more HTMLLogFiles
from within a BuildStep to hold something derived from that Step:

class MyStep(step.Step):
    def createSummary(self, log):
        html = "<html><body><p>some stuff here</p></body></html>\n"
        self.addHTMLLog("results", html)

In this case, the Waterfall display would include a link named 'results' to
this string of HTML. Perhaps this would be useful?


cheers,
 -Brian