[Buildbot-devel] nightly build only if source changes
Gaige, Yves
yves.gaige at hp.com
Tue May 20 12:56:40 UTC 2008
Hi,
Here is below a Scheduler I wrote for our own usage and that achieve the request
(I think).
For the moment it is a modified version of the Nightly scheduler where I have
added methods addChange, addImportantChange and addUnimportantChange, and made
some modifications in the __init__, doPeriodicBuild methods.
Basically it is a merge between Scheduler and Nightly, but for the moment it is
not compatible with existing Nightly. I have no time now to build a patch of
Nightly, but the remaining stuff to make it a real patch of existing is quite
straightforward:
- modify init to add 1 param: this param would control the behaviour; either
only time driven --default to be compatible with existing implementation;
or time + change (new behaviour)
- modify doPeriodicBuild: check this param 1st to choose the old or new behaviour.
BR
Yves
Yves Gaigé | OpenCall Software | yves.gaige at hp.com | +33 476 14 6330
###############################################################################
class NewNightly(BaseUpstreamScheduler):
"""Imitate 'cron' scheduling. This can be used to schedule a nightly
build, or one which runs are certain times of the day, week, or month.
Pass some subset of minute, hour, dayOfMonth, month, and dayOfWeek; each
may be a single number or a list of valid values. The builds will be
triggered whenever the current time matches these values. Wildcards are
represented by a '*' string. All fields default to a wildcard except
'minute', so with no fields this defaults to a build every hour, on the
hour.
For example, the following master.cfg clause will cause a build to be
started every night at 3:00am::
s = NewNightly('nightly', ['builder1', 'builder2'], hour=3, minute=0)
c['schedules'].append(s)
This scheduler will perform a build each monday morning at 6:23am and
again at 8:23am::
s = NewNightly('BeforeWork', ['builder1'],
dayOfWeek=0, hour=[6,8], minute=23)
The following runs a build every two hours::
s = NewNightly('every2hours', ['builder1'], hour=range(0, 24, 2))
And this one will run only on December 24th::
s = NewNightly('SleighPreflightCheck', ['flying_circuits', 'radar'],
month=12, dayOfMonth=24, hour=12, minute=0)
For dayOfWeek and dayOfMonth, builds are triggered if the date matches
either of them. All time values are compared against the tuple returned
by time.localtime(), so month and dayOfMonth numbers start at 1, not
zero. dayOfWeek=0 is Monday, dayOfWeek=6 is Sunday.
"""
compare_attrs = ('name', 'builderNames',
'minute', 'hour', 'dayOfMonth', 'month', 'dayOfWeek',
'branch', 'fileIsImportant')
def __init__(self, name, builderNames, minute=0, hour='*',
dayOfMonth='*', month='*', dayOfWeek='*',
branch=None, fileIsImportant=None):
# Setting minute=0 really makes this an 'Hourly' scheduler.
# This seemed like a better default than minute='*',
# which would result in a build every 60 seconds.
BaseUpstreamScheduler.__init__(self, name)
# Check the list of Builders
#
errmsg = ("The builderNames= argument to Scheduler must be a list "
"of Builder description names (i.e. the 'name' key of the "
"Builder specification dictionary)")
assert isinstance(builderNames, (list, tuple)), errmsg
for b in builderNames:
assert isinstance(b, str), errmsg
self.builderNames = builderNames
# The time management
#
self.minute = int(minute)
self.hour = int(hour)
self.dayOfMonth = dayOfMonth
self.month = month
self.dayOfWeek = dayOfWeek
# Build management
self.delayedRun = None
self.nextRunTime = None
self.reason = ("The NewNightly scheduler named '%s' triggered this build" % name)
# Branch / Change management
self.branch = branch
self.importantChanges = []
self.unimportantChanges = []
self.fileIsImportant = None
if fileIsImportant:
assert callable(fileIsImportant)
self.fileIsImportant = fileIsImportant
def addTime(self, timetuple, secs):
return time.localtime(time.mktime(timetuple)+secs)
def findFirstValueAtLeast(self, values, value, default=None):
for v in values:
if v >= value: return v
return default
def setTimer(self):
self.nextRunTime = self.calculateNextRunTime()
self.delayedRun = reactor.callLater(self.nextRunTime - time.time(),
self.doPeriodicBuild)
def startService(self):
log.msg("NewNightly Scheduler <%s>: Starting" % self.name)
BaseUpstreamScheduler.startService(self)
self.setTimer()
def stopService(self):
log.msg("NewNightly Scheduler <%s>: Stopping" % self.name)
BaseUpstreamScheduler.stopService(self)
self.delayedRun.cancel()
def isRunTime(self, timetuple):
def check(ourvalue, value):
if ourvalue == '*': return True
if isinstance(ourvalue, int): return value == ourvalue
return (value in ourvalue)
if not check(self.minute, timetuple[4]):
#print 'bad minute', timetuple[4], self.minute
return False
if not check(self.hour, timetuple[3]):
#print 'bad hour', timetuple[3], self.hour
return False
if not check(self.month, timetuple[1]):
#print 'bad month', timetuple[1], self.month
return False
if self.dayOfMonth != '*' and self.dayOfWeek != '*':
# They specified both day(s) of month AND day(s) of week.
# This means that we only have to match one of the two. If
# neither one matches, this time is not the right time.
if not (check(self.dayOfMonth, timetuple[2]) or
check(self.dayOfWeek, timetuple[6])):
#print 'bad day'
return False
else:
if not check(self.dayOfMonth, timetuple[2]):
#print 'bad day of month'
return False
if not check(self.dayOfWeek, timetuple[6]):
#print 'bad day of week'
return False
return True
def calculateNextRunTime(self):
return self.calculateNextRunTimeFrom(time.time())
def calculateNextRunTimeFrom(self, now):
dateTime = time.localtime(now)
# Remove seconds by advancing to at least the next minute
dateTime = self.addTime(dateTime, 60-dateTime[5])
# Now we just keep adding minutes until we find something that matches
# It not an efficient algorithm, but it'll *work* for now
yearLimit = dateTime[0]+2
while not self.isRunTime(dateTime):
dateTime = self.addTime(dateTime, 60)
#print 'Trying', time.asctime(dateTime)
assert dateTime[0] < yearLimit, 'Something is wrong with this code'
return time.mktime(dateTime)
def listBuilderNames(self):
return self.builderNames
def getPendingBuildTimes(self):
# TODO: figure out when self.timer is going to fire next and report
# that
if self.nextRunTime is None: return []
return [self.nextRunTime]
def doPeriodicBuild(self):
# Schedule the next run
self.setTimer()
if len(self.importantChanges) > 0:
changes = self.importantChanges + self.unimportantChanges
# And trigger a build
log.msg("NewNightly Scheduler <%s>: triggering build" % self.name)
bs = buildset.BuildSet(self.builderNames,
SourceStamp(changes=changes),
self.reason)
self.submit(bs)
else:
log.msg("NewNightly Scheduler <%s>: skipping build - No important change" % self.name)
# Reset the change lists
self.importantChanges = []
self.unimportantChanges = []
def addChange(self, change):
if change.branch != self.branch:
log.msg("NewNightly Scheduler <%s>: ignoring change %d on off-branch %s" % (self.name, change.revision, change.branch))
return
if not self.fileIsImportant:
self.addImportantChange(change)
elif self.fileIsImportant(change):
self.addImportantChange(change)
else:
self.addUnimportantChange(change)
def addImportantChange(self, change):
log.msg("NewNightly Scheduler <%s>: change %d from %s is important, adding it" % (self.name, change.revision, change.who))
self.importantChanges.append(change)
def addUnimportantChange(self, change):
log.msg("NewNightly Scheduler <%s>: change %d from %s is not important, adding it" % (self.name, change.revision, change.who))
self.unimportantChanges.append(change)
###############################################################################
-----Original Message-----
From: Eveque, Philippe
Sent: mardi 20 mai 2008 14:41
To: Gaige, Yves
Subject: FW: [Buildbot-devel] nightly build only if source changes
Yves,
as discussed because you already have an implementation of this
Phil.
-----Original Message-----
From: buildbot-devel-bounces at lists.sourceforge.net [mailto:buildbot-devel-bounces at lists.sourceforge.net] On Behalf Of Bailey, Darragh
Sent: mardi 20 mai 2008 13:23
To: Dustin J. Mitchell
Cc: buildbot-devel at lists.sourceforge.net
Subject: Re: [Buildbot-devel] nightly build only if source changes
I'll give it a shot.
My initial thoughts are that it should be based on the scheduler class with the addition that it should include the parameter NotBefore, which specifies a time that the build should not start before. After that time it should be started in accordance of treeStableTimer.
>From what I can tell, this should mean that the only function to be overridden is addImportantChange.
--
Regards,
Darragh Bailey
Systems Software Engineer
Hewlett Packard Galway Ltd.
+353 91 75-4674
Postal Address: Hewlett Packard Galway Limited, Ballybrit Business Park, Galway
Registered Office: Hewlett Packard Galway Limited, 63-74 Sir John Rogerson's Quay Dublin 2
Registered Number: 361933
_______________________________________________
The contents of this message and any attachments to it are confidential and may be legally privileged. If you have received this message in error you should delete it from your system immediately and advise the sender.
To any recipient of this message within HP, unless otherwise stated you should consider this message and attachments as "HP CONFIDENTIAL".
> -----Original Message-----
> From: djmitche at gmail.com [mailto:djmitche at gmail.com] On Behalf Of Dustin
> J. Mitchell
> Sent: 19 May 2008 20:12
> To: Bailey, Darragh
> Cc: buildbot-devel at lists.sourceforge.net
> Subject: Re: [Buildbot-devel] nightly build only if source changes
>
> On Mon, May 19, 2008 at 1:24 PM, Bailey, Darragh <dbailey at hp.com> wrote:
> > The question about creating a scheduler that runs nightly only if source
> changes have been made since the last build has arisen before and its
> something that I need to add to the buildbot currently used by my team.
> >
> > Has anyone already created such a class?
>
> If not, are you interested in writing one and contributing it?
>
> Dustin
>
> --
> Storage Software Engineer
> http://www.zmanda.com
-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Buildbot-devel mailing list
Buildbot-devel at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/buildbot-devel
More information about the devel
mailing list