[Buildbot-devel] stuck behind a firewall...

[Brian Warner]

> something that'a bit annoying though: some of the URLs seem to be 
> absolute URLs

> Is there a reason why it's this way? Can I make buildbot use relative 
> URLs somehow?

The short answer is that it's a bug. I'm hoping to fix it with the "web
parts" project (the modular WebStatus thing I wrote about a few weeks ago),
but it could probably also be fixed in the existing Waterfall class with a
little bit of work.

The root cause is that I didn't set up the URL hierarchy quite right. You'll
notice that Buildbot, Builders, Builds, Steps, and LogFiles form five levels
of a tree, which maps directly to the URL namespace hierarchy:

 http://example.org/
  overall buildbot page

 http://example.org/ascend-quick
  single Builder page with status and 'force build' buttons

 http://example.org/ascend-quick/builds/41
  single Build page, shows reason, blamelist, changes, steps, results

 http://example.org/ascend-quick/builds/41/step-compile
  single BuildStep page, shows logfiles (not very useful)

 http://example.org/ascend-quick/builds/41/step-compile/0
  single LogFile page, has log contents

Each of these pages is generated by a class in buildbot.status.html,
specifically the render() methods of WaterfallStatusResource,
StatusResourceBuilder, StatusResourceBuild, StatusResourceBuildStep, and
HTMLLog, respectively. Each node of the URL hierarchy corresponds to an
instance of one of these classes at the appropriate level, generated
on-demand when the URL is parsed.

The problem is that I should have made all of these URLs except the LogFile
one end in a slash. If this were the case then when, e.g., the Builder page
is constructing links to the BuildSteps and LogFiles contained within it, it
could simply emit relative URLs like "step-compile" and "step-compile/0".
Instead, it has to go through some hoops to find out what its own name is, so
it can construct a relative URL like "41/step-compile/0". These hoops were
too hard in some places, so I punted and used the absolute-URL-creating
function getURLForThing(), which is really intended for use in places like
the email sender and the IRC bot.

The real problem is that I didn't understand how relative URLs worked at the
time, nor how to use the various twisted.web utility functions like
childLink() and siblingLink(). I can't claim to understand them much better
now, alas :).

So the way to fix this is probably to scan through html.py and replace all
uses of getURLForThing() with proper relative URLs, after first changing the
URL hierarchy to let everything end in a slash. There is an 'addSlash'
attribute in the Resource class that these all inherit from which I think
uses some redirects to make sure that if you hit, e.g.,
'ascend-quick/builds/41', you'll be redirected to 'ascend-quick/builds/41/'
before the page is actually rendered. This would let the parent page emit
sane-looking URLs (without the slash) while making sure that relative URLs
generated from the Build's page wind up being children of that page instead
of siblings.

For 'web parts', I'm thinking of putting each type of status page in a
separate branch of the URL tree. (basically everything you can get to right
now under Waterfall will be available through internal pages with URLs like
'_builds/ascend-quick/41' and '_logs/ascend-quick/41/step-compile/0', and all
the new status pages I described in the docs that I posted will contain links
to these pages to provide the "detailed view"). But, well, maybe if I figure
out the slash thing then this wouldn't be necessary. Not sure yet.

It's probably no more than a day's work to fix it, if you were so inclined.
I've been bitten by the exact same bug when logging into a network remotely
and using ssh tunnels to look at a buildbot there, so such a fix would be
well received :-).

cheers,
 -Brian