[Buildbot-devel] Setup of client's environment from client

Thu May 19 19:52:24 UTC 2011

this is the ideal approach: keep as much as possible in the checked out
build tree itself.

On Thu, May 19, 2011 at 12:48 PM, Amber Yust <ayust at yelp.com> wrote:

> Makefiles and make targets are useful things for having setup like this
> coupled with the actual process.
>
> Ideally, your ShellCommands would look like this:
>
> ShellCommand(['make', 'test'])
>
> ~Amber
>
> On Thu, May 19, 2011 at 12:45 PM, Philippe McLean <
> philippe.mclean at gmail.com> wrote:
>
>> my approach would be
>>
>> - reduce the number of required environment variables to a minimum
>> (relative to some base path is convenient)
>> - set build slave global environment variables on the build slave itself.
>> That becomes part of your build slave setup. Usually unavoidable on windows.
>> - create a bat/py front end in the source tree that does per build
>> environment and execution (even better if your developers use this same
>> script/setup)
>> - use buildbot step environment variables for customization on each step
>> - keep master.cfg as simple as possible
>> - upgrade buildbot
>>
>> Every build step displays its environment in its log, so it's easy to find
>> out what environment was created.
>>
>> On Thu, May 19, 2011 at 12:32 PM, Trask <trask at techsoft3d.com> wrote:
>>
>>> On 05/19/2011 12:17 PM, Philippe McLean wrote:
>>>
>>>> what are some examples of things you need to configure in the
>>>> environment?
>>>>
>>>> On Thu, May 19, 2011 at 12:14 PM, Trask <trask at techsoft3d.com
>>>> <mailto:trask at techsoft3d.com>> wrote:
>>>>
>>>>
>>>>    * Buildbot: 0.8.3p1
>>>>    * Twisted: 10.1.0
>>>>    * Jinja: 2.5.2
>>>>    * Python: 2.6.6 (r266:84292, Sep 15 2010, 16:22:56) [GCC 4.4.5]
>>>>    * Buildmaster platform: linux2
>>>>
>>>>    In order to follow best practices I am working to capture all
>>>>    configuration information within the source itself so that it is
>>>>    versioned, tagged, etc right along with everything else.
>>>>
>>>>    One particular part has been giving me problems: how to set up the
>>>> slave
>>>>    environment based upon code it has checked out in a buildstep. If I
>>>> set
>>>>    everything in master.cfg on the buildmaster, everything works fine,
>>>>    however that does not capture the configuration data within the
>>>>    ever-changing source.
>>>>
>>>>    My original idea was to run a "setup-environment.py" script which
>>>> would
>>>>    always be run prior to the other build steps. This doesn't work
>>>> because
>>>>    it is only modifying the environment for it's own process. Even if it
>>>>    did modify the global environment that is not ideal due to likely
>>>>    collisions with simultaneous usage of the machine.
>>>>
>>>>    My next attempt was to encapsulate every command within a "setup-env"
>>>>    script. You pass the command to run to the script as an argument and
>>>>    then it sets up the environment and then runs the command. I've
>>>> gotten
>>>>    this to work to a degree but is clearly not the proper way to
>>>> accomplish
>>>>    what I am trying to do.
>>>>
>>>>    BuildProperties seems promising but I haven't thought of a clean way
>>>> to
>>>>    use them. I could imagine running the "setup-env" script, having it
>>>>    spit out the proper environment settings to use and then using
>>>>    "SetProperty" with a custom function to set Properties which THEN get
>>>>    translated into environment settings on the client. Hmmph.
>>>>
>>>>    This seems like an issue that would crop up for a lot of people so I
>>>> am
>>>>    interested in hearing how other people have solved this.
>>>>
>>>>    Thanks for any help!
>>>>
>>>>    ~Trask
>>>>
>>>>
>>> Primarily it is 3rd party SDK locations, such as:
>>>
>>> MAXSDK_2011_INSTALL_DIR=C:\build\resources\3dsmax\3dsmax2011\maxsdk
>>>
>>> PARASOLID_INSTALL_DIR=C:\build\resources\parasolid\parasolid_23.0.156\intel_nt\base
>>>
>>> Most everything else that has traditionally used environment variables
>>> could be provided on the command line to the build scripts.  These SDK dirs,
>>> however, need to be configured in the environment for the builds to work.
>>>
>>> There are about 15 like this which each vary slightly upon the compiler
>>> and 32 vs. 64 bit build.  Despite this variance, *these* will be the same on
>>> any machine that does that same build.  However, there are other directories
>>> that will always be different on a per-machine basis.
>>>
>>> I *could* do all of this setup on the master since the location of
>>> everything is planned, but then that would mean I would be keeping
>>> configuration data separate from the source itself.
>>>
>>> This isn't necessarily wrong but I have felt like the best long-term
>>> solution is to keep the configuration with the source.  I guess this will
>>> always have the flaw of the buildbot source moving on as well... Like: in 3
>>> years if I want to build today's source, will I use "Buildbot 2014" to build
>>> it or use a versioned install of Buildbot from May 19th 2011?  Does anyone
>>> else plan for this sort of possibility?
>>>
>>> ~Trask
>>>
>>
>>
>>
>> ------------------------------------------------------------------------------
>> What Every C/C++ and Fortran developer Should Know!
>> Read this article and learn how Intel has extended the reach of its
>> next-generation tools to help Windows* and Linux* C/C++ and Fortran
>> developers boost performance applications - including clusters.
>> http://p.sf.net/sfu/intel-dev2devmay
>> _______________________________________________
>> Buildbot-devel mailing list
>> Buildbot-devel at lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/buildbot-devel
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://buildbot.net/pipermail/devel/attachments/20110519/5e2e1f89/attachment.html>