[Buildbot-commits] [Buildbot] #2324: Please add the ability to automatically delete unused build directories

[Buildbot]

#2324: Please add the ability to automatically delete unused build directories
------------------------+--------------------------------------
Reporter:  smat         |      Owner:
    Type:  enhancement  |     Status:  new
Priority:  major        |  Milestone:  undecided
 Version:  master       |   Keywords:  unused build directories
------------------------+--------------------------------------
 My team runs a large buildbot install (currently 143 builders across 4
 platforms), with frequent addition and removal of builds. We find that
 accumulation of unused build directories on the slaves creates disk usage
 problems. It would be excellent if Buildbot were to automatically delete
 these directories.

 I have patched the slaves for our most platforms with the most rapidly
 changing build definitions (RHEL6 and Win7) to do this, however our
 implementation(s) are probably not suitable for direct inclusion in
 Buildbot as they are platform-specific and do not include a configuration
 mechanism to enable/disable automatic deletion. Unfortunately we do not
 have time at present to put together a clean general implementation for
 submission.

 Example trivial implementation for *NIX (not reliable on Windows due to
 inability to delete open files, importing shutil not shown):
          for d in os.listdir(self.basedir):
 -            if os.path.isdir(os.path.join(self.basedir, d)):
 +            path = os.path.join(self.basedir, d)
 +            if os.path.isdir(path):
                  if d not in wanted_dirs:
                      log.msg("I have a leftover directory '%s' that is not
 "
 -                            "being used by the buildmaster: you can
 delete "
 -                            "it now" % d)
 +                            "being used by the buildmaster: deleting now"
 +                            % d)
 +                    shutil.rmtree(path)
          return retval

 The Windows implementation includes a shutil error handler that retries
 deletion up to a fixed number of attemps with a small delay between
 attempts, and logs cases where this approach is unsuccessful. This is
 necessary on Windows for a variety of reasons including security software
 temporarily holding files open.

-- 
Ticket URL: <http://trac.buildbot.net/ticket/2324>
Buildbot <http://buildbot.net/>
Buildbot: build/test automation