[Buildbot-devel] Changing the master config dynamically

[Bernhard Urban]

why is it terrible? ;-)

this solution is used in production for over one year now, with about three
reconfigs per day on average. by now we are close to 350 different
builders.  it works quite okay :)

-Bernhard
On Oct 9, 2014 11:44 PM, "Pierre Tardy" <tardyp at gmail.com> wrote:

> con: terrible hack. No offence, but seriously, don't do that, you are
> killing kitten :)
>
>
> Pierre
>
> On Thu, Oct 9, 2014 at 3:26 PM, Bernhard Urban <lewurm at gmail.com> wrote:
>
>>
>> On Thu, Oct 9, 2014 at 2:33 PM, Pierre Tardy <tardyp at gmail.com> wrote:
>>
>>> In this case, if you reload step1 before common, then step1 will be
>>> reloaded with previous version of common.py's BaseStep. Obviously, you will
>>> very quickly encounter lots of issues, and reloading everything is a tree
>>> ordering problem combined quite slow python reflexivity mechanisms. Short
>>> story: avoid it!
>>
>>
>>
>> we are using a different approach and rename loaded modules. at the top
>> of our master.cfg we have:
>>
>> # this will reload all python files in the 'scan' directories relative to
>> master (do not use . or ..)
>> scan = ["", "step", "builders", os.path.join("step", "bench")]
>>
>> master = os.path.abspath(os.path.dirname(__file__))
>> files = filter(lambda f : f.endswith(".py") and not '__init__' in f,
>> itertools.chain(*map(lambda s : map(lambda f : os.path.join(s,f),
>> os.listdir(os.path.join(master, s))), scan)))
>> for f in files:
>>     modname = f[:-3].replace(os.sep, '.')
>>     if sys.modules.has_key(modname):
>>         oldModule = sys.modules.pop(modname)
>>         oldId = 1
>>         oldKey = None
>>         while True:
>>             oldKey = modname + "-" + str(oldId)
>>             if sys.modules.has_key(oldKey):
>>                 oldId += 1
>>             else:
>>                 break
>>         sys.modules[oldKey] = oldModule
>>         log.msg("Reloading " + modname + ", old module renamed as " +
>> oldKey)
>>
>>
>> pro: no explicit reload required after imports.  dependencies shouldn't
>> be a problem too, since it's only a rename in the dict, but the reference
>> stays the same.
>> con: a reconfig leaks a bit of memory, so every now and then you should
>> restart the buildbot master (although I don't know if reload() is any
>> better in that regard).
>>
>>
>> -Bernhard
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://buildbot.net/pipermail/devel/attachments/20141010/0f704ace/attachment.html>