[Buildbot-devel] modules, reconfig, lazy_reload & a better alternative?

[Kay Hayen]

Hello there,

So you can in theory become relatively sure that it worked. But if it
> doesn't, you will have a really hard time to find out why not.
>
> Can you explain how that is?  All the modules in the project are marked
> for reload.
>

If somethings hold a reference to your module, then it will be impossible
to collect. I remember to have seen cases where classes that inherit from
"object", and are defined locally, cannot be collected by the garbage
collector, so they can keep the module alive as well.

I will admit that if you write the code, and it doesn't do spectacular
things, that may not happen, but isn't it easy to provide a callback
function or class to Buildbot, that could remain.

So when I load a for the first time afterward, it's reloaded.
> When a loads b, b is reloaded.
>
> What exactly is broken?  It all seems to work correctly.
>

Things that are working correctly are not broken. It just takes different
amounts of luck to have it stay that way.


>
> I like the gc.collect & weak references.  I'll use it to double-check.
>



> So I would advice to give up on module reloading, and prefer to use
>> separate data only config files.
>> I recently published yamltypes which is the solution I use internally in
>> my masters.
>>
>> https://pypi.python.org/pypi/yamltypes/1.0
>>
>> Idea is that all things that you need to reconfigure are put in the yaml
>> files, not in python. reconfig will just load new version of yaml files,
>> and the problem is solved..
>> If you need a change to the python logic, then you'll need a restart.
>>
>
> Thinking of that, I am generating factories in my python code, so that
> would never work, unless I add a level of indirection, generating the
> config files instead. Then it would be interesting, should Buildbot provide
> some sort of standard configuration format that a general factory reads and
> created Buildbot objects from.  But I suppose there ain't such a thing.
>
> Is it a hypothetical it won't work or have you tried it & determined that
> it doesn't work?
>

My configuration is a bunch of "for" loops and "if" statements that call
factory functions to create buildbot objects. So turning that into a
configuration is damn near impossible.

Therefore I would like a facade of Buildbot, that does that job. Spit out a
configuration a format for what I did to the objects, that I could plug in
there. I would continue to use the same API, largely (maybe just split off
callbacks provided to Buildbot), and be immediately happy.

It's a lot of work to do this I suppose. I takes some form of serialization
of Buildbot objects created in master.cfg that is not needed currently. I
hope for it to happen one day.

Then I would still configure with code, and have a factory for that
configuration file that is 95% compatible to normal API.

Yours,
Kay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://buildbot.net/pipermail/devel/attachments/20150125/c2364959/attachment.html>