[Buildbot-devel] modules, reconfig, lazy_reload & a better alternative?

[Kay Hayen]

Hello there,

2015-01-24 10:17 GMT+01:00 Pierre Tardy <tardyp at gmail.com>:

> Hi Vitali,
> I have some experience in trying hard at reloading python modules, and
> this has been very painful for me.
> The main problem with reload it looks like your PoC does not take in
> account is load order.
>
> if a imports b, and a is reloaded before b, then a holds a reference to
> the *old* version of b.
> This ends up with lots of weird behavior and bugs.
>

One would also have to call "gc.collect()" to make sure the unused stuff
gets flushed. Using a "weakref", you can check, if a module really goes
away. It would be supposed to raise an error when used afterwards.

So you can in theory become relatively sure that it worked. But if it
doesn't, you will have a really hard time to find out why not.


> So I would advice to give up on module reloading, and prefer to use
> separate data only config files.
> I recently published yamltypes which is the solution I use internally in
> my masters.
>
> https://pypi.python.org/pypi/yamltypes/1.0
>
> Idea is that all things that you need to reconfigure are put in the yaml
> files, not in python. reconfig will just load new version of yaml files,
> and the problem is solved..
> If you need a change to the python logic, then you'll need a restart.
>

Thinking of that, I am generating factories in my python code, so that
would never work, unless I add a level of indirection, generating the
config files instead. Then it would be interesting, should Buildbot provide
some sort of standard configuration format that a general factory reads and
created Buildbot objects from.  But I suppose there ain't such a thing.

But generally, code as configuration is very powerful, and the main point
that hooks me on Buildbot. But if there were a configuration format with
code supporting its creation, it would be even sweeter.

Ideally, I would only change my "imports" to another name space, run my
buildbot configuration code, and it would output stuff, that can be used
loaded and updated to a  master.cfg.

Just my 2cents, no way I would implement this myself, just a wish.

Yours,
Kay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://buildbot.net/pipermail/devel/attachments/20150124/06e67dd4/attachment.html>