[Buildbot-devel] Setup of client's environment from client

[Philippe McLean]

my approach would be

- reduce the number of required environment variables to a minimum (relative
to some base path is convenient)
- set build slave global environment variables on the build slave itself.
That becomes part of your build slave setup. Usually unavoidable on windows.
- create a bat/py front end in the source tree that does per build
environment and execution (even better if your developers use this same
script/setup)
- use buildbot step environment variables for customization on each step
- keep master.cfg as simple as possible
- upgrade buildbot

Every build step displays its environment in its log, so it's easy to find
out what environment was created.

On Thu, May 19, 2011 at 12:32 PM, Trask <trask at techsoft3d.com> wrote:

> On 05/19/2011 12:17 PM, Philippe McLean wrote:
>
>> what are some examples of things you need to configure in the environment?
>>
>> On Thu, May 19, 2011 at 12:14 PM, Trask <trask at techsoft3d.com
>> <mailto:trask at techsoft3d.com>> wrote:
>>
>>
>>    * Buildbot: 0.8.3p1
>>    * Twisted: 10.1.0
>>    * Jinja: 2.5.2
>>    * Python: 2.6.6 (r266:84292, Sep 15 2010, 16:22:56) [GCC 4.4.5]
>>    * Buildmaster platform: linux2
>>
>>    In order to follow best practices I am working to capture all
>>    configuration information within the source itself so that it is
>>    versioned, tagged, etc right along with everything else.
>>
>>    One particular part has been giving me problems: how to set up the
>> slave
>>    environment based upon code it has checked out in a buildstep. If I set
>>    everything in master.cfg on the buildmaster, everything works fine,
>>    however that does not capture the configuration data within the
>>    ever-changing source.
>>
>>    My original idea was to run a "setup-environment.py" script which would
>>    always be run prior to the other build steps. This doesn't work because
>>    it is only modifying the environment for it's own process. Even if it
>>    did modify the global environment that is not ideal due to likely
>>    collisions with simultaneous usage of the machine.
>>
>>    My next attempt was to encapsulate every command within a "setup-env"
>>    script. You pass the command to run to the script as an argument and
>>    then it sets up the environment and then runs the command. I've gotten
>>    this to work to a degree but is clearly not the proper way to
>> accomplish
>>    what I am trying to do.
>>
>>    BuildProperties seems promising but I haven't thought of a clean way to
>>    use them. I could imagine running the "setup-env" script, having it
>>    spit out the proper environment settings to use and then using
>>    "SetProperty" with a custom function to set Properties which THEN get
>>    translated into environment settings on the client. Hmmph.
>>
>>    This seems like an issue that would crop up for a lot of people so I am
>>    interested in hearing how other people have solved this.
>>
>>    Thanks for any help!
>>
>>    ~Trask
>>
>>
> Primarily it is 3rd party SDK locations, such as:
>
> MAXSDK_2011_INSTALL_DIR=C:\build\resources\3dsmax\3dsmax2011\maxsdk
>
> PARASOLID_INSTALL_DIR=C:\build\resources\parasolid\parasolid_23.0.156\intel_nt\base
>
> Most everything else that has traditionally used environment variables
> could be provided on the command line to the build scripts.  These SDK dirs,
> however, need to be configured in the environment for the builds to work.
>
> There are about 15 like this which each vary slightly upon the compiler and
> 32 vs. 64 bit build.  Despite this variance, *these* will be the same on any
> machine that does that same build.  However, there are other directories
> that will always be different on a per-machine basis.
>
> I *could* do all of this setup on the master since the location of
> everything is planned, but then that would mean I would be keeping
> configuration data separate from the source itself.
>
> This isn't necessarily wrong but I have felt like the best long-term
> solution is to keep the configuration with the source.  I guess this will
> always have the flaw of the buildbot source moving on as well... Like: in 3
> years if I want to build today's source, will I use "Buildbot 2014" to build
> it or use a versioned install of Buildbot from May 19th 2011?  Does anyone
> else plan for this sort of possibility?
>
> ~Trask
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://buildbot.net/pipermail/devel/attachments/20110519/024c4082/attachment.html>