[Buildbot-devel] buildbot caching config file

[Greg Ward]

On 20 October 2008, Jimmy John said:
> I have a buildbot setup that runs every night at 1:00 am. One of the
> steps is to pull down a log file from the slave and place it in a dir
> corresponding to the current date. e.g

Are you sure you need to do this?  Buildbot is very good at keeping
track of build logs, getting them from the slave to the master, serving
them up via HTTP, etc.  Fundamentally, Buildbot is just an engine for 1)
the master telling the slave "run this command" and 2) getting the
output of that command back to the master.  Sounds like you are
partially reinventing this wheel.

(If you have build steps that write to their own log files rather than
stdout/stderr, there is even a way to tell Buildbot about it, so it can
track/transfer those logs too.  I can't remember if it's a simple
constructor arg to BuildStep or something more involved, but I believe
it is documented.)

Finally, remember that every log tracked by Buildbot has its own URL
that will return a copy of the log via HTTP.

> The problem is that when buildbot starts up, it seems to have computed
> destfilename_part correctly, but it caches it. So every night, the
> reports get dumped into the directory corr to the date when buildbot
> master was started. It does not compute destfilename_part afresh at
> each run.

Right, because your "config" file is just a bunch of Python code that is
executed *once*, when Buildbot starts.  (This is a common source of
confusion.)  You need a way to defer execution of that
datetime.datetime() call.  Offhand, I'd say you're better off avoiding
the whole situation simply by taking advantage of Buildbot's
infrastructure for dealing with build logs.

        Greg