[Buildbot-devel] Dynamic list of logfiles

[Greg Ward]

On 16 February 2009, Bill Baker said:
> I have half of that -- the slave automatically posts files to an external
> web server, and the BuildStep notices in stdio and adds a link.  Straight
> from master.cfg:
> 
> class LogFileShellCommand(ShellCommand):
>     log_files = []
>     log_files.append("config.log")
>     log_files.append("build.log")
> 
>     def __init__(self, **kwargs):
>         # And upcall to let the base class do its work
>         ShellCommand.__init__(self, **kwargs)
> 
>     def createSummary(self, log):
>         for line in log.getText().split("\n"):
>             for log_file in self.log_files:
>                 # line format is "    log file build.log saved to http://..."
>                 if (line.startswith("log file ") and line.find(" saved to ") >= 0):
>                     split = line.split(" saved to ", 1)
>                     self.addURL(split[0].split("log file ")[1], split[1])

What a charming hack!  As long as the slave command cooperates and
prints something like "logging to foo.log", you can take advantage of
that and suss out the log file URLs or filenames.  (BTW, you could
make your code shorter and arguably clearer with a regex there.)

You inspired me to try something similar but slightly more powerful:
  * use a LogLineObserver to watch for those "logging to foo.log" lines
    in my slave command's stdout
  * when I see one, turn it into a LogFile and start watching it

If this had worked, it would have been super cool and would have done
everything I wanted.  Alas!  This appears to be the first nifty hack
I've tried with Buildbot that falls down flat, and it's because the
slave-side code doesn't let you add new log files while a command is
running.  Specifically, buildbot.slave.command.ShellCommand's
attribute logFileWatchers list is populated in the constructor and not
changed again.  So there is no way for the master to tell the slave
"start watching foo.log now".  Darn.

I suspect that dynamic log file nirvana requires a protocol change.
Sod it.  I'll see if I can figure out a way to do this after the step
completes.

       Greg