[Buildbot-devel] modules, reconfig, lazy_reload & a better alternative?

[Vitali Lovich]

> On Jan 24, 2015, at 1:48 AM, Kay Hayen <kay.hayen at gmail.com> wrote:
> 
> Hello there,
> 
> 2015-01-24 10:17 GMT+01:00 Pierre Tardy <tardyp at gmail.com <mailto:tardyp at gmail.com>>:
> Hi Vitali,
> I have some experience in trying hard at reloading python modules, and this has been very painful for me.
> The main problem with reload it looks like your PoC does not take in account is load order.
> 
> if a imports b, and a is reloaded before b, then a holds a reference to the *old* version of b.
> This ends up with lots of weird behavior and bugs.
> 
> One would also have to call "gc.collect()" to make sure the unused stuff gets flushed. Using a "weakref", you can check, if a module really goes away. It would be supposed to raise an error when used afterwards.
> 
> So you can in theory become relatively sure that it worked. But if it doesn't, you will have a really hard time to find out why not.
Can you explain how that is?  All the modules in the project are marked for reload.

So when I load a for the first time afterward, it’s reloaded.
When a loads b, b is reloaded.

What exactly is broken?  It all seems to work correctly.

I like the gc.collect & weak references.  I’ll use it to double-check.

>  
> So I would advice to give up on module reloading, and prefer to use separate data only config files.
> I recently published yamltypes which is the solution I use internally in my masters.
> 
> https://pypi.python.org/pypi/yamltypes/1.0 <https://pypi.python.org/pypi/yamltypes/1.0>
> 
> Idea is that all things that you need to reconfigure are put in the yaml files, not in python. reconfig will just load new version of yaml files, and the problem is solved..
> If you need a change to the python logic, then you'll need a restart.
> 
> Thinking of that, I am generating factories in my python code, so that would never work, unless I add a level of indirection, generating the config files instead. Then it would be interesting, should Buildbot provide some sort of standard configuration format that a general factory reads and created Buildbot objects from.  But I suppose there ain't such a thing.
Is it a hypothetical it won’t work or have you tried it & determined that it doesn’t work?

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://buildbot.net/pipermail/devel/attachments/20150124/21f5b487/attachment.html>