[Buildbot-devel] BuildBot & Visual Studios....

[BRM]

----- Original Message -----

> From: Dmitry Nezhevenko <dion at dion.org.ua>
> On Fri, Sep 23, 2011 at 01:33:30PM +0200, Benoît Allard wrote:
>>  We should make the current steps more configurable. Add a parameter
>>  to decide on which build binary should be invoked (msdev, devenv,
>>  vcbuild or msbuild), and then smartly build the command line based
>>  on the above choice from the same set of parameters looks like the
>>  best approach to me.
> From my point of view, it's better to have dedicated VCBuild(), DevEnv()
> and MSBuild() steps. They can have common base class...

Either way - it should be explicit which to use, however it is implemented in the backend.
The simplest way, is probably to use - as you say - separate steps for each, but this also complicates the API a little.

Consider:

factory.addStep(VS2003(projectfile='MyProject.vcproj', config='Release', mode='rebuild'))

How would this change?

Do we really want to have:

factory.addStep(VS2003MsEnv(projectfile='MyProject.vcproj', config='Release', mode='rebuild'))
factory.addStep(VS2003DevEnv(projectfile='MyProject.vcproj', config='Release', mode='rebuild'))
factory.addStep(VS2003VCBuild(projectfile='MyProject.vcproj', config='Release', mode='rebuild'))
factory.addStep(VS2003MSBuild(projectfile='MyProject.vcproj', config='Release', mode='rebuild'))

Or is it simpler to simply have: 

factory.addStep(VS2003(projectfile='MyProject.vcproj', config='Release', mode='rebuild',buildtool='msenv'))
factory.addStep(VS2003(projectfile='MyProject.vcproj', config='Release', mode='rebuild',buildtool='devenv'))
factory.addStep(VS2003(projectfile='MyProject.vcproj', config='Release', mode='rebuild',buildtool='vcbuild'))
factory.addStep(VS2003(projectfile='MyProject.vcproj', config='Release', mode='rebuild',buildtool='msbuild'))

I think the second makes more sense. It also maintains compatibility, and we can set a sane default.
It also makes it easier to report when a certain versions of VS stop providing one tool or update to add another.

Ultimately the user should not care so long as the build gets done correctly, so the default will most likely be utilizes 90+% of the time.

>>  I also agree with Ben about not relying too much on env variables,
>>  I'd rather pick the installdir from the registry.
> Why? It's pretty standard way to retrieve install directory from scripts. 

Don't rely on environment variables for finding the appropriate BAT file. Use the registry.
It's cleaner, simpler, and more reliable.
 
>>  Same, I rather set
>>  the right env variables myself than relying on vcvarsall.bat, that
>>  way everything is kept under buildbot's control and can be exactly
>>  replayed again.
> 
> vcvarsall.bat is also documented way to do this:
>     http://msdn.microsoft.com/en-us/library/x4d2c09s(v=vs.80).aspx
> 
> If we're talking that everything should be under buildbot's control, 
> then
> it probably should unpack VS (or even reinstall OS) for every build...

We're talking around a different issue here.
The real issue in the above is: do we want to have buildBot setup and teardown the vs environment?
The question is: is that really necessary?

Now, to setup the environment it is best to rely on the BAT files provided by MS with each version of VS.
To tear down, you have to UNSET about 10 environment variables.
This can be done cleanly (I've done it in my own scripting environment); but you have to do research for every version of VS that is supported.
(I have information for VS6, 2003, and 2008 at present.)

However, I doubt that is necessary for buildBot to do unless two builds running on the same slave share the same environment variables.
If they do, then the better solution would be to try to separate those two environments so that they don't interfere with each other.

>>  Dimitry, can you send me the output of the "set" command once you
>>  ran vcvarsall.bat from VC9 (with both possible parameters) ? I
>>  thought I had it installed over there, and cannot find it ... VC9
>>  and VC10 from vstudio.py needs an update on that point.
> It's actually not two possible cases... There is also ia64, x86_amd64,
> x86_ia64. VS2011 preview already contains three "arm" variants.
> 
> The first issue with vstudio.py is that on 64bit versions of Windows, even
> for drive "c:", VS will be installed to Program Files (x86).

And locating the installation based on the registry will resolve that issue too.
 
> Anyway I'm attaching diff between original env and after vcvarsall for
> "x86" and "x64" architectures. But please, do NOT try to 
> emulate vcvarsall
> behavior (like doing same changes). vcvarsall may be changed. It's
> behavior also changes if WinSDK in installed.
> 
> vcvarsall is _documented_ and _official_ way to get compiler shell on
> Windows.

100% agreed.
 
----- Original Message -----
> From: Dmitry Nezhevenko <dion at dion.org.ua>
> On Fri, Sep 23, 2011 at 08:40:21AM -0400, Marc-Antoine Ruel wrote:
>>  Le 22 septembre 2011 23:15, Dustin J. Mitchell <dustin at v.igoro.us> a 
> écrit :
>> 
>>  > As I mentioned in #2108, I'm not at all opposed to Benoit's 
> suggestion
>>  > to add a query-the-registry command on the slave side, and use it
>>  > master-side to determine necessary paths.
>> 
>> 
>>  Often, I feel like a small wrapper python script that would be shelled out
>>  would be *much simpler* to manage than the overhead of:
>>  - Add registry query implementation and unit tests
>>  - Add server-side logic to change VS exe path depending on the registry key
> 
> Guys, vcvarsall.bat is not just about registry. It also adds include
> paths and some other environment variables (even modifies it if they
> already exists).
> 
> It may happens that there are scripted build steps in projects that
> depends on that _standard_ environment. That's why I've hacked original
> vstudio.py to use proper way for getting compiler shell.
> 
> If you want "sane" solution that will work as expected for all projects,
> the only way to set environment is to call vcvarsall.bat (or emulate
> everything it have, that is error prone).

VCVARSALL/VCVAR32 does nothing with the registry itself. It merely sets up a command shell environment for accessing the various tools for a specific version of VS.
And yes, it is the proper way to setup such an environment - a method that buildBot should rely on.

However, finding VCVARSALL/VCVARS32 should not rely on environment variables or hard coded paths.
The best way to locate the installation is via the registry.

What I wrote about in the original bug report (and started this whole thread with) has nothing to do with setting up the actual environment as it does locating VCVARSALL/VCVARS32.bat.
I simply want to locate the installation via the registry, and to that effect I will do the work necessary to do so.

Ben