[Buildbot-devel] Changing the master config dynamically

[Bernhard Urban]

On Thu, Oct 9, 2014 at 2:33 PM, Pierre Tardy <tardyp at gmail.com> wrote:

> In this case, if you reload step1 before common, then step1 will be
> reloaded with previous version of common.py's BaseStep. Obviously, you will
> very quickly encounter lots of issues, and reloading everything is a tree
> ordering problem combined quite slow python reflexivity mechanisms. Short
> story: avoid it!



we are using a different approach and rename loaded modules. at the top of
our master.cfg we have:

# this will reload all python files in the 'scan' directories relative to
master (do not use . or ..)
scan = ["", "step", "builders", os.path.join("step", "bench")]

master = os.path.abspath(os.path.dirname(__file__))
files = filter(lambda f : f.endswith(".py") and not '__init__' in f,
itertools.chain(*map(lambda s : map(lambda f : os.path.join(s,f),
os.listdir(os.path.join(master, s))), scan)))
for f in files:
    modname = f[:-3].replace(os.sep, '.')
    if sys.modules.has_key(modname):
        oldModule = sys.modules.pop(modname)
        oldId = 1
        oldKey = None
        while True:
            oldKey = modname + "-" + str(oldId)
            if sys.modules.has_key(oldKey):
                oldId += 1
            else:
                break
        sys.modules[oldKey] = oldModule
        log.msg("Reloading " + modname + ", old module renamed as " +
oldKey)


pro: no explicit reload required after imports.  dependencies shouldn't be
a problem too, since it's only a rename in the dict, but the reference
stays the same.
con: a reconfig leaks a bit of memory, so every now and then you should
restart the buildbot master (although I don't know if reload() is any
better in that regard).


-Bernhard
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://buildbot.net/pipermail/devel/attachments/20141009/4af7a6ed/attachment.html>