[Buildbot-devel] Beta is out!
Dustin J. Mitchell
dustin at zmanda.com
Thu Mar 11 07:00:55 UTC 2010
I promised Chris I'd have a beta out on Wednesday, and it's still
technically Wednesday on the Pacific coast of the US.
The first beta is tagged at
http://github.com/djmitche/buildbot/tree/v0.8.0beta1
http://github.com/djmitche/buildbot/tarball/v0.8.0beta1
http://github.com/djmitche/buildbot/zipball/v0.8.0beta1
And testing would be *very* much appreciated! Please file bugs for
any problems you find, marked with version 0.8.0. However, a big
caution is in order: WARNING: there is no support for downgrading
from 0.8.0 to 0.7.x! Do not upgrade a production buildmaster if you
are not prepared to ride out the bugs you encounter!
After installing the new version, you will need to run 'buildbot
upgrade-master' to migrate your Changes into the new database.
New features in this release:
- uses an SQLite database to store change information
- Jinja templates
- Support for multiple projects in one buildmaster
- most tests no longer run automatically - we're starting over with
working, isolated tests.
- reorganized documentation
(http://djmitche.github.com/buildbot/docs/0.8.0beta1)
The first of those changes affected a *lot* of code, and likely
introduced some bugs we haven't found yet. Please tease those out
with us!
Attached is the git shortlog since the last release. I'd like to
particularly thank Brian Warner, who of course wrote the new DB-backed
scheduler system; Marcus Lindblom, who almost singlehandedly did the
Jinja2 conversion; Chris Atlee, who has been attacking and solving
some incredibly deep bugs; Kovarththanan Rajaratnam, who fixed an
enormous number of bugs and small errors and generally made Buildbot a
lot more presentable; and Benoit Allard, who has probably drafted his
repository patch ten times now at my behest (it's landed!).
I'd like to do beta releases on a weekly basis until I'm satisfied
that we've got a working application, at which point I'll switch to
release candidates, and we'll be at 0.8.0!
Dustin
----
Amar Takhar (2):
BitKeeper hook script, based off of svn_buildbot.py.
Add a BK (BitKeeper) source step for retrieving soirce via bk.
Aviv Ben-Yosef (1):
Small tidying to use existing method
Benoit Allard (5):
add Visual Studio compile steps
Add System32 to VC8 PATH, cmd is needed
Add the possibility to run Mercurial Change Hook as a forked process
Restore consistency in table.sql
Add the project and repository property to some changesource
Brad Hards (1):
Add try --get-builder-names option
Brian Warner (24):
rpmspec.py: fix failure in use of 'types', tidy up imports
test_vc: fix hg shortids, make CVS_checkout_options not clobber CVS
status/web/baseweb.py: buildbot_css moved (fix test_web, test_config)
test_vc.CVS_checkout_options: use "-n" instead of "-N" to make
both "cvs checkout" and "cvs export" happy
util.py: add DictOfSets (with tests), LRUCache, defaultdict (for py2.4)
loop.py: a notification-triggered processor-loop handling class
eventual.py: add eventual-send operator, copied (with tests) from Foolscap
schedulerdb 1-of-5: main database code, create/upgrade-DB in runner.py
schedulerdb 2-of-5: Changes
schedulerdb 3-of-5: BuildRequest/BuildSet
schedulerdb 4-of-5: Schedulers
schedulerdb 5-of-5: Builders, everything else
update all tests to work with the new schedulerdb
p4poller: Clean up Deferred errbacks, log.err problems, fix
tests to ignore logged errors when testing bad input.
hush most non-trivial pyflakes warnings
add code-coverage tools: use "make test-coverage" to get started
turn off execute bit for all buildbot/* python modules
db.py: fix get_sqlite_dbapi_name() to actually use "sqlite3" on >=py2.6
test_config: rewrite in terms of MasterMixin
remove every-30-second SchedulerManager trigger, in favor of one
post-reconfig trigger
hush some pyflakes warnings
add c['db_poll_interval'], to enable multiple-masters with a shared DB
Charles Lepple (2):
Manual: explain re.match vs re.search in Compile step
Add to description of WarningCountingShellCommand
Chase Phillips (4):
Add changeAdded status notifications.
Update text in Source and ShellCommand's status on hookup.
Fix a nit in TreeSize.
buildstatus should only show steps that have started.
Chris AtLee (22):
Allow per-builder configuration of buildHorizon, logHorizon, and
eventHorizon
Allow per-builder configuration of buildHorizon, logHorizon, and
eventHorizon
Adding slaveConnected / slaveDisconnected hooks for status plugins
Adding tests for slaveConnected / slaveDisconnected
Fix checks for nextSlave / nextBuild hooks.
Adding DB.from_url method and tests
Refactor database loading code so that it can be specified via master.cfg
Prevent changing the db_url on a reconfig
Updating create-master/upgrade-master to use DB urls
Reorganize buildbot.slave.commands
Fix exception if slave admin isn't defined
Fixing recursion problems in loop processing
Move setText out of setStepStatus into startStep.
Catch exceptions in ShellCommand.describe, log them, and then carry on
Remove newlines from chunks macro
Make step elapsed time use util.formatInterval
Display '' for slaves with no admin set on buildslaves page
Try and treat log files as UTF-8 encoded, but fallback to
replacing unknown characters with placeholders.
Fix display of not-yet-run steps, and standardize ETA display
Buffer data sent to master
Working on tests and comments
Imporve test coverage of slave.commands.base
Dustin J. Mitchell (204):
(fixes #662) add Jinja2 to setup.py
fix display of a builder with no recent builds (don't display
the build table at all)
fix grid display of unbuilt revisions (previously resulted in traceback)
add .mailmap to help shortlog
add comment for command_version 2.9
fix typos
remove bogus tests
(fixes #679) reference 'status.web.auth' in docs
fix error showing unfinished builds
(fixes #649) whitespace commit to check github commit hook
(fixes #649) commit to check github commit hook
(fixes #686) always do a clobber when retrying a VC operation
add a path_to_root key so that directory listings render
properly (fixes #687)
remove some unnecessary arch and darcs files; add other files to
MANIFEST.in; add test for this sort of thing
updates to MANIFEST.in to include all *new* files
bring NEWS up to date
fix some tests .. but not all
remove test_mergerequests.py
update NEWS from release
(refs #691) don't use chagnes in the customHTMLMailMessage for the moment
Revert "remove test_mergerequests.py"
add description and descriptionDone parameters to
MasterShellCommand (fixes #690)
remove test_limitlogs completely
remove 'as' keyword
compare mercurial repos without trailing slashes (fixes #694)
remove slavebuilder images, too
update CREDITS
fix buildbot.status.words to import even if PyOpenSSL isn't installed
fix a few test failures on windoze
reorganize tests into subdirectories
fix typo
(fixes #683) add extra error-checking for bogus inputs to addStep
fix distribution stuff to correspond to new test file locations
fix whitespace dependency in test_console
fixed missed file move
remove more unused scripts
fix syntax error (no base class for ContextMixin) in
buildbot/status/web/base.py:150
Remove buildbot.test.runs.test_run.BuildPrioritization.testPriority
skip interrupt tests on windows instead of TODO
remove debugging code
wait for builds to finish before going onto the next test (fixes
dirty reactor errors)
don't assume that deferreds have fired - wait for them
add note about order_console_by_time
(fixes #679) render a helpful message when !DirectoryLister isn't avilable
don't TODO tests for interrupts on windows
don't try to reset $HOME if it wasn't set to begin with
make test_webparts run without 'net access
add path to DTD files in setup.py
use chmod -Rf
use 'find $foo -exec chmod u+rwx {} ;' on FreeBSD
test that clobber worked the first time, extra debug info
add 'revision' to changes created by SyncmailMaildirSource
Fix console to not misrepresent older builds
ignore twisted/ too
Jinjafy json status view
move merged test scripts into new subdirectories
Document JSON view
update NEWS for next release
require simplejson on <Python-2.6
Don't fail if getChangeForBuild gets None for a build
add additional diagnostic information to simplejson deps
Try just unconditionally requiring simplejson
go back to just adding simplejson to the reqs in 2.5 and below
remove deprecated html.Waterfall
remove some dead code
allow individual control of all web-based actions
avoid unreferenced variable in build list
make notifications a bit more resilient to missing methods
remove unused LastBuild and getLastNBuilds from baseweb.py
factor OneBoxPerBuilder and OneLinePerBuild out into modules
Replace /builders with contents of /one_box_per_builder
use path_to_foo() functions instead of hand-crafted relative paths
split documentation into per-chapter files
add pointer to API docs
Support building docs in both split and full HTML
move 'Change Sources' under 'Configuration'
break out 'Schedulers' section, add indices for each section
WIP - look for comment to see where I was
fix simplejson import in buildbot.db
require pysqlite on py-2.5 and lower, use it in buildbot.db
clean up test_db output a bit
more test output cleanup
fix test_webparts (I broke it..)
comments on buildbot.loop
clean up log display
use textwrap.dedent to make schema more readable
don't fail if lxml not installed
Use Versioned to ensure SourceStamps have the right types
requestBuild no longer exists; uses submitBuildRequest
remove redundant print
Make buildbot.changes.changes.Change() more resilient
check for notification methods before calling them
Finish moving docs around
get images right in docs
Add (stubbed-out) Developer Information section
Fix bad 'authz' configuration in sample.cfg
fix requirement that BuilderConfig category be non-None
disable badly isolated test
remove bogus testing code
clean up buildbot.test.runs.test_db.Scheduling's isolation
disable badly-isolated
buildbot.test.runs.test_db.{Create,MigrateChanges,Generic} (refs #724)
move all tests to buildbot.broken_test (refs #725)
MANIFEST fixup
remove broken_test from the dist
test_util_ComparableMixin (refs #725)
rename test to reflect package path
add tests for buildbot.db.DB (refs #725)
move unit tests to unit/, since we will have integration tests too
rename 'DB' to the more descriptive 'DBSpec'
remove unused 'generic' table and functions
better naming for authz tests, only do one assert per test
fix thread leak, fix sqlite errors on shutdown, add tests
add tests for DBConnection.parmlist
buildbot.util.eventual - move, test, document
use the correct name for the moved module
buildbot.util.loop, new tests, buildbot.test.state
move test_util_loop to buildbot.test.unit
test open_db, create_db, create_or_upgrade_db; fix leak
add buildbot.util to the packages list
fix console bug in first/last conditionals
Revert buildbot.util.collections addition
fix missed change in import location
add and test buildbot.util.collections
remove dead code (CancelableDeferred, ingoreStaleRefs)
finish testing and documenting buildbot.util
use a regular \uXXXX unicode character reference
fix outdated import
fix test_web
P4Sync bugfix (felo on github)
break buildbot.db out into packages
add buildbot.test.unit to the dist, since buildbot.test is
already included
fix leftover buildbot.db references
separate dbspec, connector, and schema; add schema upgrade mechanism
remove bogus import
correctly include tables.sql
don't start master unless schema is_current
make test.unit.test_db_connector less sensitive to other things
leaking threads
make a base class for Upgraders
document new DB stuff
make test_db_dbspec's thread-leak-protection better, and stop
leaking threads
add forgotten db-changing step
include buildbot.slave.commands in install
fix recursion-proofing
add some comments to tables.sql
revert d04983d3a5168a5f4362d1602d790b56fed05ce6 and use jinja's
- to collapse whitespace
add developer note about jinja whitespace
add docs for service hierarchy
simple, incomplete tests for buildbot.schedulers.basic.Scheduler
Be very careful in how JSON support is imported
add project, repository columns to changes and sourcestamps tables
add project and repository to Change objects and SourceStamps,
display them
display SourceStamps correctly
add repository and project properties
add a generic change filter class and test it
fix test failures
add filtering on project and repository
document some of project and repository
fix NameError when config initialization fails
use correct default for repository and project in sendChange
fix filtering
more repo/proj documentation
also exclude /tgrid in robots.txt
(refs #725) update NEWS regarding newly-passing buildbot tests
fill in Jinja NEWS
add version numbers
fix typo
try (unsuccesfully) to exclude broken_test from epydoc
Gareth Armstrong (2):
Add documentation fixes for MailNotifier messageFormatter
function and add example code.
Ian Zimmerman (1):
defaultMessage should be a standalone function, not a method of
MailNotifier
Jochen Eisinger (2):
Implement filtering events by committers. This allows for
restricting the waterfall view to events generated by specific
committers.
Limit the display to 24h when a branch or committer filter is
specified for the waterfall (and not other limit is explicitly given).
Jonathan S. Romero (1):
Update documentation for always_purge option to SVN buildstep.
Kovarththanan Rajaratnam (76):
Move Mike Taylor to CREDITS
Make ZopeInterface dependency more visible/explicit
Remove Mike Taylors duplicate entry
Split into _finished_ok/failure
P4Poller (test): change to use move/add (feature introduced in
Perforce 2009.1)
P4Poller: handle new Perforce 2009.1 operations
Makefile.sample: add some error checking
setup: place web stuff correctly
MANIFEST.in: robots.txt already covered by buildbot/status/web/files/*
MANIFEST.in: static files are placed under buildbot/status/web/files/*
P4Poller: pass through important environment variables (fixes #625)
README: Add Jinja2 to the list of requirements
README: require python-2.4 or later due to Jinja2
Fix line endings on windows files (no functionality change)
Try: remove statement which reference an unknown attribute
Try.getopt() only accepts 3 arguments
Test: remove 'substring in string' workaround for < 2.3
TryClient: add Perforce support (fixes #258)
Advertise the newly added Perforce TryClient
PerforceExtractor.getBaseRevision(): quote arguments properly
PerforceExtractor: raise an exception if we couldn't extract the
changelist number
PerforceExtractor: require that the user specify a branch
PerforceExtractor: document limitation and usage
Remove email.MIMEMultipart workaround for versions less than python-2.2.2
Remove bz2.BZ2File workaround for versions less than python-2.3
Remove gzip.GzipFile workaround for versions less than python-2.4
set is a builtin type in python-2.4
Rename scripts/tryclient.py -> clients/tryclient.py
Remove dnotify.py which is completly unused
Yet another dnotify.py that must die ;)
Test: fix a couple of newlines issues on Windows
Doc: clarify that extraRecipients is a list of strings
Lowercase directory names for consistency
Pass along string to master:port
Remove the main loop. We already support this through 'buildbot statuslog'
Epydocify TextClient constructor
Add support for username/passwd in TextClient
Accept username/passwd as command line arguments in 'buildbot statuslog'
Point to refs #658
Move asDict() next to html_dict() (no functionality change)
Add (simple)json requirement
Move into correct subfolders
Streamline master/slave .tac files so that the common parts
appears first (no functionality change intended)
Add new sqlite3 dependency
import signal unconditionally
Remove superfluous dbgMsg()
Add getPendingBuildTimes() to Base scheduler
Fixup some comments in ChangeManager
Remove __main__. We already support this through 'buildbot debugclient'
(fixes #719) fix what appears to be some debug code that was
inadvertently left in
Fix some bad indentions
Remove duplicate etree import
Add a short CONTRIBUTE section
Highlight that we've removed support for 'bots' and 'sources'
Raise an error instead of a warning if we see c['bots'] or c['sources']
Don't even mention that we used to support 'bots' and 'sources'.
We don't want to give people some wrongs ideas
Be a model citizen. Cleanup our sample config files
Scrub contrib/ for c['sources']
Remove c['bots'] backwards compatible testcase which is no
longer applicable
Remove c['sources'] backwards compatible testcase which is no
longer applicable
Fix thinko in previous commit. Of cource I meant c['slaves']
Indent last build column properly (whitespace change)
(fixes #647) Use basedir to form the path to the twistd.log file
Remove stale comment wrt. Python 2.3 compatibility
Group Python stdlib imports
BuilderConfig category must be a string
Triggerable doesn't do anything useful with get_initial_state()
so don't override it
debugclient has been moved to buildbot.clients
Point to http://buildbot.net/trac in the API reference docs
M-A (5):
Add asDict() member function to many status classes
Add /json status.
Add test_sourcestamp to test sourcestamp.SourceStamp.
Add status_json basic unit test.
Messed alignement when I reformated the file, causing incorrect behavior.
Marcus Lindblom (186):
First step on using Jinja template engine: the about page
Made footer into template + minor cleanup
ChangeSources now templated + added common footer to most views
Changed BuildSlaves page into jinja template
Moved BuildSlaves page to Jinja template + remove double footer
on Slave page
Converted buildslave page to Jinja template
Changed BuildStep view into Jinja template
Moved Builders list into Jinja template
Rewrote 'build' page as jinja template
Rewrote builder page as jinja template (+ fix for build page)
Oneboxperbuilder -> jinja
waterfall & waterfall help partially converted to jinja
Removed accidentally commited buildstatus.py from aborted merge
Jinja use i18n and trim_blocks, plus get_template fix for footer
webstatus.grid/tgrid -> jinja
Adding Eclipse files to .gitignore
Removed status.web.tests (old code that wasn't used)
authfail -> jinja
fix build page error (added missing string conversion)
removed remaining references to status.web.tests module
fix jinja template access in all html resources (via
req.site.buildbot_service)
removed /tests/ url from docstring
feeds/rss -> jinja + fixes
feeds/atom -> jinja
feeds xml cleanup
Unify feeds content generation (via template selection)
Cleanup feed code (use single xml per format, simplify
FeedResource significantly)
No need to replace \n with <br/> when wrapped in <pre>
onelineperbuild & onelinemixin -> jinja
forms -> jinja + minor slave page fix
buildslave.html should use build_line() macro
Rename OneLineMixin -> BuildLineMixin
started on root page, not enabled yet
Fixed minor issues causing web tests to fail
Add builder name to make webparts test pass + add _trial_temp*
to .gitignore
Make use-of-undefined-var-in-jinja2-template as errors and fix
all issues + minor improvs + TODO cleanup
buildstatus -> jinja
Add missing change to waterfallhelp.html
Add missing change to build.html
Quick fix to console view when revision is None
Fix syntax error in slave view
Fix 'view in waterfall' link on builder page
Fix syntax errors in import
CRLF dummy commit
Replace index.html root with jinja template (fixes #656) + use
new default.css everywhere
Attempt to jinjaify HTML log. Not working yet
IBox HTML -> jinja macro (fixes #652) + use path_to_build(er) in grid
All templates now inherits layout.html (Fixes #655)
Convert StaticHTML and LastBuild to use jinja
utf-8 encode all log output. Fixes #653
Changes HTML -> jinja (fixes #654)
Remove unused index.html (superceded by templates/root.html)
Fixed some indentation (keep at multiples of 4)
Move public_html/index.html to templates/root.html on
'upgrade-master' (fixes #663)
Decode log contents to unicode from utf-8, not ascii
Cleanup all unused imports and tag the rest as ok (for Eclipse/PyDev)
Transfer files in strict binary mode & close tarfile fd
Convert list contents to strings before hashing (fixes
test_locks on win32)
test_ec2: Fix bad change in
ca08c45cfbb773fabe4d49ce060bb6f96583b480 and allow for win32
USERPROFILE env
More warning silencing
test_limitlogs: don't assume we can run scripts directly
Console view -> jinja (superficial parts only) (see #651)
Web status cleanups (fix #665) + more
Fix some failing web tests
Console -> jinja (complete) (fixes #651) + cleanups
Add top menu to layout template + minor html cleanups
Remove leftover console_html.py (console is fully jinjaified)
Revert "test_limitlogs: don't assume we can run scripts directly"
Webstatus cleanup/fixes + layout improvements (columns,
alternate colour of rows, etc)
More HTML tweaks & fixed (css, list -> table, padding adjustment)
Minor console fixes (index error in code with sort-by-time +
show category names in html properly)
HTML: Buildslave lists -> table + gradient & css tweak
Fix some python 2.4 incompabilities
Fix file sorting in changes view
Add missing css_class to unstarted buildsteps
Improve layout of build and buildstep HTML
Fix Home link when buildbot is not shown from root and the
current url doesn't hold a /
About page: add alternating colors and wrap in column
one_line_per_build: accentuate 'all builders' in force form +
layout improvements
buildslaves.html: add escaping and fix improve spam protection of e-mail
Show slave host info only on slave-page, not in slave-list nor
on builder page
Decode slave host & admin data as utf-8 (fixes #671)
List relevant builders per slave in buildsslaves view (fixes #673)
Mercurial: Retrieve full 40-char rev id instead of short
Add & apply jinja filter for username, email and revision + minor tweaks
Move static web files to own subdir of buildbot.status.web
Hide full-revs and emails by default
Comment out console view debuginfo
Don't add extra markup if short rev == long rev + use divs
Add changecommitlink arg to WebStatus (add bug links in change comments)
Add revlink arg to WebStatus (adds links to revisions-ids) + cleanups
Document revlink & changecommentlink in manual
tryclient: retrieve full id for Mercurial + sync docs
Move HtmlRsource.getContext to new ContextMixin-class that
HtmlResource & DirectoryLister inherits. (fixes #688)
Align coding style and css-classes in directory.html with other templates
Remove now-irrelevant comment in getContext()
CSS: Use min-width & em instead of width & pixels + minor formatting
Add WebpartsRecursive: follow all links from rot and do
xhtml/dtd validation
Extract jinja-env setup and fix CustomHTMLMail test (fixes #691)
WebPartsRecursive test: add more builders/builds/logs, test # of
pages/links and fix issues
Disable accidentally commited log enable
Get Application data directoy using win32-api rather than
deprecated env-var.
Strip whitespace and quotes to allow string comparisons to
succeed on windows
Add ability to set title attribute on WebStatus
changecommentlink:s (third string in tuple)
Console view: Move some html from python to jinja-template + tweaks
Fix broken console test
Fix HTML validation error (replace two spaces at a time instead)
Mattias Brändström (1):
Using unicode strings where appropriate.
Michael MacDonald (2):
add SSL support to IRC bot
* fix broken useSSL patch
Mook (1):
add /grid to WebStatus robots.txt
Nicol√°s Alvarez (11):
Add missing close tag in waterfall.
box_macros: Leave spaces between HTML attributes.
Remove spurious </span> in build_line.html.
Fix mismatched HTML tag in builder page.
Fix unclosed <input> tag in buildslave.html
Use lowercase 'post' in <form method="post">.
build.html: Add missing quotes around attribute values.
change_macros.html: Fix closing td tag.
Fix mismatched and missing tags in buildstep.html
Validation fixes to grid_macros.html
Change how the "last heard from" column in /buildslaves is rendered.
Satyagraha 1956 (1):
Fix for #676 .
Stefan Seefeld (10):
Fix minor issue in waterfall display.
Install templates.
* Allow template overrides in buildbot instance directory. *
Access templates through HtmlResource base class.
* Initialize jinja globally, not once per HtmlResource instance.
* Introduce layout template for others to extend. * Let WaterfallHelp
use the new content() method.
Move the jinja environment into the WebStatus object.
Fix the way the jinja templates are accessed in HTMLResource
Add template support for directory views.
Use layout.html for waterfall template.
Use layout.html for slaves views.
Use layout.html for builder views.
Tim Hatch (4):
Fix formatting for pending builds
Document that tryclient works with bzr and git
Make tryclient docs for darcs more concise
Shift tryclient docs for hg/darcs to be consistent
Tom Fogal (2):
Remove tabs(!).
Fix directory listings with multiple files.
gv (1):
- Replaced documentation figures * The slavebuilder figure
was merged with the master figure * Added an auto-crop step to
images/Makefile
--
Open Source Storage Engineer
http://www.zmanda.com
More information about the devel
mailing list