[users at bb.net] buildbot retains memory of an earlier location of a buildslave?
Greg Bullock
greg at nwra.com
Thu Jun 16 17:19:35 UTC 2016
It dawns on me that the problem may not be that the buildbot somehow
retains a memory of the earlier location of a buildslave. It could
instead be that the entire log that displays is actually from an earlier
(and failed) build attempt.
Is there a way to have buildbot insert a datetime stamp into the log for
a given step?
On 6/14/2016 3:01 PM, Greg Bullock wrote:
> I'm using version 0.8.12.
>
> I have indeed rebooted the master. I've also deleted the CentOS
> virtual machine (the slave) and created a new one from a fresh install
> of the system. No luck.
>
> I have not tried both of these measures in the same test, I wouldn't
> think that could make the difference.
>
> I will look into purging the sun cache, from the log, it seems that
> the buildslave is already reporting using the obsolete (and
> nonexistent) working directory before it calls svn.
>
> Greg Bullock
> NorthWest Research Associates
> 301 Webster St. <geo:0,0?q=301%20Webster%20St.%20Monterey%20CA%2093940>
> Monterey, CA 93940 <geo:0,0?q=301%20Webster%20St.%20Monterey%20CA%2093940>
> (831) 582-4907 <tel:%28831%29%20582-4907>
> greg at nwra.com
>
> -------- Original message --------
> From: Pierre Tardy <tardyp at gmail.com>
> Date: 6/14/16 1:12 PM (GMT-08:00)
> To: Greg Bullock <greg at nwra.com>, users at buildbot.net
> Subject: Re: [users at bb.net] buildbot retains memory of an earlier
> location of a buildslave?
>
> Hi,
> Looks indeed like an interresting problem. Did you try to reboot the
> master?
>
> The master might indeed store the workdir, but I would expect it to be
> reset when the work is reconnecting.
>
> Which version of buildbot is that?
>
> Also thing to check is svn. could svn cache something?
>
> Regards
> Pierre
>
> Le mar. 14 juin 2016 à 21:04, Greg Bullock <greg at nwra.com
> <mailto:greg at nwra.com>> a écrit :
>
> Where might the buildbot retain its memory of an earlier location of a
> buildslave? I've moved a buildslave to a different drive, but the log
> from the new build tests shows the steps (e.g., the SVN, ShellCommand,
> and Compile steps) all running from the original drive, not the
> new drive.
>
> My buildbot runs on Windows 7 64-bit, and this particular buildslave
> (called "centos7-worker") runs on a virtual machine on CentOS 7
> (linux)
> hosted on the same Windows machine with the buildbot master.
>
> Originally, the buildslave's files were on a shared drive (which
> is also
> a Windows share shared from the Windows host), in
> /media/sf_G_DRIVE/buildbot/centos7-worker, but the SVN checkout step
> would successfully checkout only about half the project, then give
> an error
>
> ------------------------------------------------------------
> ... (previous lines omitted)
> svn: E200030: disk I/O error, executing statement 'RELEASE s10246'
> svn: E200030: sqlite: disk I/O error
> svn: E200030: sqlite: disk I/O error
> svn: E200030: no such savepoint: s10247, executing statement
> 'RELEASE s10247'
> svn: E200030: no such savepoint: s10247, executing statement
> 'ROLLBACK TO s10247'
> program finished with exit code 1
> ------------------------------------------------------------
>
> After a Google search turned up several mentions of conflicts between
> Windows shared drive and sqlite (see, for example,
> http://stackoverflow.com/questions/18782941/tortoisesvn-checkout-sqlite-disk-i-o-error-s10),
> I moved the buildslave files to a local drive within the virtual
> machine, with the path /home/buildbot/Documents/centos7-worker.
>
> At this point, I expect the log to have references only to the new
> /home/buildbot location, with no more references to the older
> /media/sf_G_DRIVE location. But instead, the log still shows steps
> running "in dir /media/sf_G_DRIVE/buildbot/centos7-worker/...".
> The top
> of the log from the SVN checkout step shows an example:
>
> ------------------------------------------------------------
> svn --version
> in dir
> /media/sf_G_DRIVE/buildbot/centos7-worker/gfortran_centos7_build/build
> (timeout 1200 secs)
> watching logfiles {}
> argv: ['svn', '--version']
> environment:
> DBUS_SESSION_BUS_ADDRESS=unix:abstract=/tmp/dbus-ebvEqaI7D1,guid=b74561f6a34d14afe5191e515759b435
> DESKTOP_SESSION=gnome-classic
> DISPLAY=:0
> GDMSESSION=gnome-classic
> GDM_LANG=en_US.UTF-8
> GJS_DEBUG_OUTPUT=stderr
> GJS_DEBUG_TOPICS=JS ERROR;JS LOG
> GNOME_DESKTOP_SESSION_ID=this-is-deprecated
> GNOME_SHELL_SESSION_MODE=classic
> GPG_AGENT_INFO=/run/user/1000/keyring/gpg:0:1
> HISTCONTROL=ignoredups
> HISTSIZE=1000
> HOME=/home/buildbot
> HOSTNAME=localhost.localdomain
> IMSETTINGS_INTEGRATE_DESKTOP=yes
> IMSETTINGS_MODULE=none
> LANG=en_US.UTF-8
> LESSOPEN=||/usr/bin/lesspipe.sh %s
> LOGNAME=buildbot
> LS_COLORS=rs=0:di=38;5;27:ln=38;5;51:mh=44;38;5;15:pi=40;38;5;11:so=38;5;13:do=38;5;5:bd=48;5;232;38;5;11:cd=48;5;232;38;5;3:or=48;5;232;38;5;9:mi=05;48;5;232;38;5;15:su=48;5;196;38;5;15:sg=48;5;11;38;5;16:ca=48;5;196;38;5;226:tw=48;5;10;38;5;16:ow=48;5;10;38;5;21:st=48;5;21;38;5;15:ex=38;5;34:*.tar=38;5;9:*.tgz=38;5;9:*.arc=38;5;9:*.arj=38;5;9:*.taz=38;5;9:*.lha=38;5;9:*.lz4=38;5;9:*.lzh=38;5;9:*.lzma=38;5;9:*.tlz=38;5;9:*.txz=38;5;9:*.tzo=38;5;9:*.t7z=38;5;9:*.zip=38;5;9:*.z=38;5;9:*.Z=38;5;9:*.dz=38;5;9:*.gz=38;5;9:*.lrz=38;5;9:*.lz=38;5;9:*.lzo=38;5;9:*.xz=38;5;9:*.bz2=38;5;9:*.bz=38;5;9:*.tbz=38;5;9:*.tbz2=38;5;9:*.tz=38;5;9:*.deb=38;5;9:*.rpm=38;5;9:*.jar=38;5;9:*.war=38;5;9:*.ear=38;5;9:*.sar=38;5;9:*.rar=38;5;9:*.alz=38;5;9:*.ace=38;5;9:*.zoo=38;5;9:*.cpio=38;5;9:*.7z=38;5;9:*.rz=38;5;9:*.cab=38;5;9:*.jpg=38;5;13:*.jpeg=38;5;13:*.gif=38;5;13:*.bmp=38;5;13:*.pbm=38;5;13:*.pgm=38;5;13:*.ppm=38;5;13:*.tga=38;5;13:*.xbm=38;5;13:*.xpm=38;5;13:*.tif=38;5;13:*.tiff=38;5;13:*.png=38;
> 5;13:*.svg=38;5;13:*.svgz=38;5;13:*.mng=38;5;13:*.pcx=38;5;13:*.mov=38;5;13:*.mpg=38;5;13:*.mpeg=38;5;13:*.m2v=38;5;13:*.mkv=38;5;13:*.webm=38;5;13:*.ogm=38;5;13:*.mp4=38;5;13:*.m4v=38;5;13:*.mp4v=38;5;13:*.vob=38;5;13:*.qt=38;5;13:*.nuv=38;5;13:*.wmv=38;5;13:*.asf=38;5;13:*.rm=38;5;13:*.rmvb=38;5;13:*.flc=38;5;13:*.avi=38;5;13:*.fli=38;5;13:*.flv=38;5;13:*.gl=38;5;13:*.dl=38;5;13:*.xcf=38;5;13:*.xwd=38;5;13:*.yuv=38;5;13:*.cgm=38;5;13:*.emf=38;5;13:*.axv=38;5;13:*.anx=38;5;13:*.ogv=38;5;13:*.ogx=38;5;13:*.aac=38;5;45:*.au=38;5;45:*.flac=38;5;45:*.mid=38;5;45:*.midi=38;5;45:*.mka=38;5;45:*.mp3=38;5;45:*.mpc=38;5;45:*.ogg=38;5;45:*.ra=38;5;45:*.wav=38;5;45:*.axa=38;5;45:*.oga=38;5;45:*.spx=38;5;45:*.xspf=38;5;45:
> MAIL=/var/spool/mail/buildbot
> OLDPWD=/home/buildbot
> PATH=/usr/lib64/qt-3.3/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/bin:/sbin:/home/buildbot/.local/bin:/home/buildbot/bin
> PWD=/media/sf_G_DRIVE/buildbot/centos7-worker/gfortran_centos7_build/build
> QTDIR=/usr/lib64/qt-3.3
> QTINC=/usr/lib64/qt-3.3/include
> QTLIB=/usr/lib64/qt-3.3/lib
> QT_GRAPHICSSYSTEM_CHECKED=1
> QT_IM_MODULE=ibus
> SESSION_MANAGER=local/unix:@/tmp/.ICE-unix/2119,unix/unix:/tmp/.ICE-unix/2119
> SHELL=/bin/bash
> SHLVL=2
> SSH_AGENT_PID=2306
> SSH_AUTH_SOCK=/run/user/1000/keyring/ssh
> TERM=xterm-256color
> USER=buildbot
> USERNAME=buildbot
> VTE_VERSION=3803
> WINDOWID=44049837
> WINDOWPATH=1
> XAUTHORITY=/run/gdm/auth-for-buildbot-xH6kQ3/database
> XDG_CURRENT_DESKTOP=GNOME-Classic:GNOME
> XDG_MENU_PREFIX=gnome-
> XDG_RUNTIME_DIR=/run/user/1000
> XDG_SEAT=seat0
> XDG_SESSION_DESKTOP=gnome-classic
> XDG_SESSION_ID=1
> XDG_VTNR=1
> XMODIFIERS=@im=ibus
> _=/usr/bin/buildslave
> using PTY: False
> svn, version 1.7.14 (r1542130)
> compiled Nov 20 2015, 19:25:09
> (subsequent lines omitted)...
> ------------------------------------------------------------
>
> I've tried recreating the buildbot master and both of its buildslaves
> (deleting the master folder and the buildslaves folders, creating the
> buildbot and buildslaves, then restoring the buildbot.tacs and
> master.cfg from backup). And I also unmounted the /media/sf_G_DRIVE
> altogether. All of this seemed to purge the log as expected, but the
> next build attempt still makes reference to the old location of the
> CentOS buildslave in /media/sf_G_DRIVE drive.
>
> Where might the reference to this obsolete location of the
> buildslave be
> stored? The centos7-worker's buildbot.tac file contains no explicit
> reference to the obsolete location:
> ------------------------------------------------------------
>
> import os
>
> from buildslave.bot import BuildSlave
> from twisted.application import service
>
> basedir = '.'
> rotateLength = 10000000
> maxRotatedFiles = 10
>
> # if this is a relocatable tac file, get the directory
> containing the TAC
> if basedir == '.':
> import os.path
> basedir = os.path.abspath(os.path.dirname(__file__))
>
> # note: this line is matched against to check that this is a
> buildslave
> # directory; do not edit it.
> application = service.Application('buildslave')
>
> try:
> from twisted.python.logfile import LogFile
> from twisted.python.log import ILogObserver, FileLogObserver
> logfile = LogFile.fromFullPath(os.path.join(basedir,
> "twistd.log"),
> rotateLength=rotateLength,
> maxRotatedFiles=maxRotatedFiles)
> application.setComponent(ILogObserver,
> FileLogObserver(logfile).emit)
> except ImportError:
> # probably not yet twisted 8.2.0 and beyond, can't set log yet
> pass
>
> buildmaster_host = 'kraken'
> port = 9989
> slavename = 'centos7-worker'
> passwd = 'pass'
> keepalive = 600
> usepty = 0
> umask = None
> maxdelay = 300
> allow_shutdown = None
>
> s = BuildSlave(buildmaster_host, port, slavename, passwd, basedir,
> keepalive, usepty, umask=umask, maxdelay=maxdelay,
> allow_shutdown=allow_shutdown)
> s.setServiceParent(application)
>
>
> Likewise, the master.cfg file creates a single SVN checkout step as
>
> checkout = SVN(repourl = 'svn://kraken/trunk/sf_code',
> username = "buildbot",
> password = "buildbot",
> haltOnFailure = True,
> )
> ------------------------------------------------------------
>
> and this step is used in all builders on both buildslaves (the
> "centos7-worker" and the "win64-worker"), so I don't see how it could
> convey the obsolete location to the buildslave.
>
>
> --
> Greg Bullock
> NorthWest Research Associates
> 301 Webster St.
> Monterey, CA 93940
> (831) 582-4907
> greg at nwra.com <mailto:greg at nwra.com>
>
>
> _______________________________________________
> users mailing list
> users at buildbot.net <mailto:users at buildbot.net>
> https://lists.buildbot.net/mailman/listinfo/users
>
>
>
> _______________________________________________
> users mailing list
> users at buildbot.net
> https://lists.buildbot.net/mailman/listinfo/users
--
Regards.
Greg Bullock
NorthWest Research Associates
301 Webster St.
Monterey CA 93940
(831) 582-4907
greg at nwra.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.buildbot.net/pipermail/users/attachments/20160616/50ffaffb/attachment.html>
More information about the users
mailing list