[Buildbot-devel] Setup of client's environment from client

[Trask]

On 05/19/2011 12:17 PM, Philippe McLean wrote:
> what are some examples of things you need to configure in the environment?
>
> On Thu, May 19, 2011 at 12:14 PM, Trask <trask at techsoft3d.com
> <mailto:trask at techsoft3d.com>> wrote:
>
>
>     * Buildbot: 0.8.3p1
>     * Twisted: 10.1.0
>     * Jinja: 2.5.2
>     * Python: 2.6.6 (r266:84292, Sep 15 2010, 16:22:56) [GCC 4.4.5]
>     * Buildmaster platform: linux2
>
>     In order to follow best practices I am working to capture all
>     configuration information within the source itself so that it is
>     versioned, tagged, etc right along with everything else.
>
>     One particular part has been giving me problems: how to set up the slave
>     environment based upon code it has checked out in a buildstep. If I set
>     everything in master.cfg on the buildmaster, everything works fine,
>     however that does not capture the configuration data within the
>     ever-changing source.
>
>     My original idea was to run a "setup-environment.py" script which would
>     always be run prior to the other build steps. This doesn't work because
>     it is only modifying the environment for it's own process. Even if it
>     did modify the global environment that is not ideal due to likely
>     collisions with simultaneous usage of the machine.
>
>     My next attempt was to encapsulate every command within a "setup-env"
>     script. You pass the command to run to the script as an argument and
>     then it sets up the environment and then runs the command. I've gotten
>     this to work to a degree but is clearly not the proper way to accomplish
>     what I am trying to do.
>
>     BuildProperties seems promising but I haven't thought of a clean way to
>     use them. I could imagine running the "setup-env" script, having it
>     spit out the proper environment settings to use and then using
>     "SetProperty" with a custom function to set Properties which THEN get
>     translated into environment settings on the client. Hmmph.
>
>     This seems like an issue that would crop up for a lot of people so I am
>     interested in hearing how other people have solved this.
>
>     Thanks for any help!
>
>     ~Trask
>

Primarily it is 3rd party SDK locations, such as:

MAXSDK_2011_INSTALL_DIR=C:\build\resources\3dsmax\3dsmax2011\maxsdk
PARASOLID_INSTALL_DIR=C:\build\resources\parasolid\parasolid_23.0.156\intel_nt\base

Most everything else that has traditionally used environment variables 
could be provided on the command line to the build scripts.  These SDK 
dirs, however, need to be configured in the environment for the builds 
to work.

There are about 15 like this which each vary slightly upon the compiler 
and 32 vs. 64 bit build.  Despite this variance, *these* will be the 
same on any machine that does that same build.  However, there are other 
directories that will always be different on a per-machine basis.

I *could* do all of this setup on the master since the location of 
everything is planned, but then that would mean I would be keeping 
configuration data separate from the source itself.

This isn't necessarily wrong but I have felt like the best long-term 
solution is to keep the configuration with the source.  I guess this 
will always have the flaw of the buildbot source moving on as well... 
Like: in 3 years if I want to build today's source, will I use "Buildbot 
2014" to build it or use a versioned install of Buildbot from May 19th 
2011?  Does anyone else plan for this sort of possibility?

~Trask