[Buildbot-devel] modules, reconfig, lazy_reload & a better alternative?

[Vitali Lovich]

> On Jan 23, 2015, at 10:38 AM, Vitali Lovich <vlovich at gmail.com> wrote:
> 
>> 
>> On Jan 23, 2015, at 10:34 AM, Mikhail Sobolev <mss at mawhrin.net> wrote:
>> 
>> Hi Vitali,
>> 
>> On Fri, Jan 23, 2015 at 10:15:55AM -0800, Vitali Lovich wrote:
>>> I was trying to find a way to keep my buildbot organized into modules like a
>>> regular project but not lose the functionality of reconfig & whatnot like one
>>> normally would.
>>> 
>>> I stumbled across  https://github.com/dabrahams/bbproto
>>> <https://github.com/dabrahams/bbproto> & the lazy-reload links.  However, I
>>> was unhappy with lazy reload as it seemed like I would have to annotate that
>>> everywhere I load one of my modules, I need to lazy-reload it.  Additionally,
>>> it’s unclear how that works well with complex dependencies (it seems like
>>> modules would keep getting reloaded multiple times which would have its own
>>> headaches & possibly bugs)
>>> 
>>> From my prototyping, this seems like a much better solution.  Are there any
>>> problems?  I haven’t tried it out with buildbot yet.
>>> 
>>> import os
>>> import sys
>>> 
>>> codebase_dir = os.path.dirname(os.path.abspath(__file__))
>>> modules_to_delete = [name for name, module in sys.modules.iteritems() if hasattr(module, '__file__') and module.__file__.startswith(codebase_dir)]
>>> for module in modules_to_delete:
>>>   del sys.modules[module]
>>> 
>>> Basically, this is in the master.cfg before I load any modules.  The
>>> master.cfg is at the top-level directory of all other python files I might
>>> import.  So the master.cfg basically unloads any modules that are in my
>>> project.
>> Meta buildbot (http://buildbot.buildbot.net) uses this approach:
>> 
>>   https://github.com/buildbot/metabbotcfg/blob/master/master.cfg#L3
>> 
>> Do you see any problems with it?  (Obivously, the "2nd" level modules would not
>> get reloaded unless they are explicitly marked so.)
> Isn’t there a concern that they get loaded twice on startup?
> 
> Also, I do have 2nd level modules.  Having to explicitly & manually reload everything is not ideal.
> 
> That being said, I’m not having much luck putting the above proto-type into practice so I may have to do this.
More specifically, my concern is my code is organized as followed (reloads added where I think they would have to be)

— master.cfg
import configuration
import configuration.webstatus
import projects
reload(configuration)
reload(configuration.webstatus)
reload(projects)

— configuration/__init__.py
import webstatus
reload(webstatus)

— projects/__init__.py
import configuration
reload(configuration)

From this setup, configuration gets reloaded something like 3 times unless I’m mistaken.
Is there a better way?

-Vitali

>> -- 
>> Misha
>> 
>> ------------------------------------------------------------------------------
>> New Year. New Location. New Benefits. New Data Center in Ashburn, VA.
>> GigeNET is offering a free month of service with a new server in Ashburn.
>> Choose from 2 high performing configs, both with 100TB of bandwidth.
>> Higher redundancy.Lower latency.Increased capacity.Completely compliant.
>> http://p.sf.net/sfu/gigenet
>> _______________________________________________
>> Buildbot-devel mailing list
>> Buildbot-devel at lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/buildbot-devel

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://buildbot.net/pipermail/devel/attachments/20150123/ed83100d/attachment.html>