[Buildbot-devel] Per-build temp directory

[Johnathan Hegge]

On Jul 6, 2010, at 4:51 PM, Greg Ward wrote:

> Hi all --
> 
> I'm wrestling with a bizarre Visual Studio problem -- every so often
> it dies claiming "Permission denied" on a file in %TEMP%.  (Naturally
> it doesn't say if it's trying to read or write the file.)  Since it's
> intermittent, I strongly suspect that "Permission denied" is a secret
> Microsoft codeword for "too many files in this directory" or "disk
> full" or "pffft, I'm just not going to work today, and you can't make
> me, nyahh nyahh".  And of course I cannot reproduce it; it only seems
> to affect builds that are needed by QA *right now*.

I believe I've seen this kind of transient error in the past.  I also think I've seen different causes in each case: service user permissions, file overwriting errors.  

One consistent issue I've encountered with Visual Studio (for production builds) is pre-compiled headers.  I don't think that's pertinent in your case but if this TEMP issue is a side-effect....  Windows build environments, always tricky and finicky.   

> 
> As a possible workaround, I came up with the idea of using a per-build
> temporary directory.  (We run each build in a brand-new directory,
> since we're using BuildBot for full production builds, not
> incrementals.)  But I can't think of a good way to do this.  I have a
> global constant in my master.cfg for the workdir:
> 
>  DEFAULT_WORKDIR = WithProperties("%s", "ims_build_id")
> 
> (ims_build_id is a custom build property that's different for each
> build.) But that's relative to the builder's directory, which is
> relative to the slave directory.  So in reality,
> 
>  workdir = <slave dir> + <builder dir> + <ims_build_id>
>          = "e:\slave" + "build.windows" + "7342"
>          = "e:\slave\build.windows\7342"
> 
> but of course that computation happens on the slave, rather late in
> the game.  I want to be able to do
> 
>  factory.addStep(ShellCommand(
>    command="windows-build"
>    env={ 'TEMPDIR': <slave dir> + <builder dir> + <ims_build_id> + "tmp" })
> 
> I don't see how to do it with WithProperties, since <slave dir> and
> <builder dir> do not seem to be exposed as build properties.  And I'm
> pretty sure that %TEMPDIR% really needs to be an absolute path.
> 
> Ideas?

I went through some similar idea, wanting to pass in a dynamic value for an environment variable.  cmd just doesn't work like a (bash, etc.) shell and I wound up just doing a wrapper script, either with ant and properties or a batch file.  Pass minimal info through WithProperties and work it out on the slave.  

Other ideas:  change TEMPDIR and TMP to some value other than the default (well, for my environ on XP) to avoid some system vs. user problems (I've seen this suggested again on MSDN blogs) or maybe use cygwin to get a shell...  

> 
> Oh yeah, I'm using BuildBot 0.7.12.  Upgrading to 0.8.x will be a bit
> of a bother, so I'll do it when I have to.
> 
> Thanks --
> 
> Greg
> 
> ------------------------------------------------------------------------------
> This SF.net email is sponsored by Sprint
> What will you do first with EVO, the first 4G phone?
> Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
> _______________________________________________
> Buildbot-devel mailing list
> Buildbot-devel at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/buildbot-devel