[users at bb.net] buildbot retains memory of an earlier location of a buildslave?

Greg Bullock greg at nwra.com
Thu Jun 16 17:19:35 UTC 2016


It dawns on me that the problem may not be that the buildbot somehow 
retains a memory of the earlier location of a buildslave.  It could 
instead be that the entire log that displays is actually from an earlier 
(and failed) build attempt.

Is there a way to have buildbot insert a datetime stamp into the log for 
a given step?



On 6/14/2016 3:01 PM, Greg Bullock wrote:
> I'm using version 0.8.12.
>
> I have indeed rebooted the master. I've also deleted the CentOS 
> virtual machine (the slave) and created a new one from a fresh install 
> of the system. No luck.
>
> I have not tried both of these measures in the same test, I wouldn't 
> think that could make the difference.
>
> I will look into purging the sun cache, from the log, it seems that 
> the buildslave is already reporting using the obsolete (and 
> nonexistent) working directory before it calls svn.
>
> Greg Bullock
> NorthWest Research Associates
> 301 Webster St. <geo:0,0?q=301%20Webster%20St.%20Monterey%20CA%2093940>
> Monterey, CA 93940 <geo:0,0?q=301%20Webster%20St.%20Monterey%20CA%2093940>
> (831) 582-4907 <tel:%28831%29%20582-4907>
> greg at nwra.com
>
> -------- Original message --------
> From: Pierre Tardy <tardyp at gmail.com>
> Date: 6/14/16 1:12 PM (GMT-08:00)
> To: Greg Bullock <greg at nwra.com>, users at buildbot.net
> Subject: Re: [users at bb.net] buildbot retains memory of an earlier 
> location of a buildslave?
>
> Hi,
> Looks indeed like an interresting problem. Did you try to reboot the 
> master?
>
> The master might indeed store the workdir, but I would expect it to be 
> reset when the work is reconnecting.
>
> Which version of buildbot is that?
>
> Also thing to check is svn. could svn cache something?
>
> Regards
> Pierre
>
> Le mar. 14 juin 2016 à 21:04, Greg Bullock <greg at nwra.com 
> <mailto:greg at nwra.com>> a écrit :
>
>     Where might the buildbot retain its memory of an earlier location of a
>     buildslave?  I've moved a buildslave to a different drive, but the log
>     from the new build tests shows the steps (e.g., the SVN, ShellCommand,
>     and Compile steps) all running from the original drive, not the
>     new drive.
>
>     My buildbot runs on Windows 7 64-bit, and this particular buildslave
>     (called "centos7-worker") runs on a virtual machine on CentOS 7
>     (linux)
>     hosted on the same Windows machine with the buildbot master.
>
>     Originally, the buildslave's files were on a shared drive (which
>     is also
>     a Windows share shared from the Windows host), in
>     /media/sf_G_DRIVE/buildbot/centos7-worker, but the SVN checkout step
>     would successfully checkout only about half the project, then give
>     an error
>
>     ------------------------------------------------------------
>        ... (previous lines omitted)
>        svn: E200030: disk I/O error, executing statement 'RELEASE s10246'
>        svn: E200030: sqlite: disk I/O error
>        svn: E200030: sqlite: disk I/O error
>        svn: E200030: no such savepoint: s10247, executing statement
>     'RELEASE   s10247'
>        svn: E200030: no such savepoint: s10247, executing statement
>     'ROLLBACK TO s10247'
>        program finished with exit code 1
>     ------------------------------------------------------------
>
>     After a Google search turned up several mentions of conflicts between
>     Windows shared drive and sqlite (see, for example,
>     http://stackoverflow.com/questions/18782941/tortoisesvn-checkout-sqlite-disk-i-o-error-s10),
>     I moved the buildslave files to a local drive within the virtual
>     machine, with the path /home/buildbot/Documents/centos7-worker.
>
>     At this point, I expect the log to have references only to the new
>     /home/buildbot location, with no more references to the older
>     /media/sf_G_DRIVE location.  But instead, the log still shows steps
>     running "in dir /media/sf_G_DRIVE/buildbot/centos7-worker/...". 
>     The top
>     of the log from the SVN checkout step shows an example:
>
>     ------------------------------------------------------------
>        svn --version
>         in dir
>     /media/sf_G_DRIVE/buildbot/centos7-worker/gfortran_centos7_build/build
>     (timeout 1200 secs)
>         watching logfiles {}
>         argv: ['svn', '--version']
>         environment:
>     DBUS_SESSION_BUS_ADDRESS=unix:abstract=/tmp/dbus-ebvEqaI7D1,guid=b74561f6a34d14afe5191e515759b435
>          DESKTOP_SESSION=gnome-classic
>          DISPLAY=:0
>          GDMSESSION=gnome-classic
>          GDM_LANG=en_US.UTF-8
>          GJS_DEBUG_OUTPUT=stderr
>          GJS_DEBUG_TOPICS=JS ERROR;JS LOG
>          GNOME_DESKTOP_SESSION_ID=this-is-deprecated
>          GNOME_SHELL_SESSION_MODE=classic
>          GPG_AGENT_INFO=/run/user/1000/keyring/gpg:0:1
>          HISTCONTROL=ignoredups
>          HISTSIZE=1000
>          HOME=/home/buildbot
>          HOSTNAME=localhost.localdomain
>          IMSETTINGS_INTEGRATE_DESKTOP=yes
>          IMSETTINGS_MODULE=none
>          LANG=en_US.UTF-8
>          LESSOPEN=||/usr/bin/lesspipe.sh %s
>          LOGNAME=buildbot
>     LS_COLORS=rs=0:di=38;5;27:ln=38;5;51:mh=44;38;5;15:pi=40;38;5;11:so=38;5;13:do=38;5;5:bd=48;5;232;38;5;11:cd=48;5;232;38;5;3:or=48;5;232;38;5;9:mi=05;48;5;232;38;5;15:su=48;5;196;38;5;15:sg=48;5;11;38;5;16:ca=48;5;196;38;5;226:tw=48;5;10;38;5;16:ow=48;5;10;38;5;21:st=48;5;21;38;5;15:ex=38;5;34:*.tar=38;5;9:*.tgz=38;5;9:*.arc=38;5;9:*.arj=38;5;9:*.taz=38;5;9:*.lha=38;5;9:*.lz4=38;5;9:*.lzh=38;5;9:*.lzma=38;5;9:*.tlz=38;5;9:*.txz=38;5;9:*.tzo=38;5;9:*.t7z=38;5;9:*.zip=38;5;9:*.z=38;5;9:*.Z=38;5;9:*.dz=38;5;9:*.gz=38;5;9:*.lrz=38;5;9:*.lz=38;5;9:*.lzo=38;5;9:*.xz=38;5;9:*.bz2=38;5;9:*.bz=38;5;9:*.tbz=38;5;9:*.tbz2=38;5;9:*.tz=38;5;9:*.deb=38;5;9:*.rpm=38;5;9:*.jar=38;5;9:*.war=38;5;9:*.ear=38;5;9:*.sar=38;5;9:*.rar=38;5;9:*.alz=38;5;9:*.ace=38;5;9:*.zoo=38;5;9:*.cpio=38;5;9:*.7z=38;5;9:*.rz=38;5;9:*.cab=38;5;9:*.jpg=38;5;13:*.jpeg=38;5;13:*.gif=38;5;13:*.bmp=38;5;13:*.pbm=38;5;13:*.pgm=38;5;13:*.ppm=38;5;13:*.tga=38;5;13:*.xbm=38;5;13:*.xpm=38;5;13:*.tif=38;5;13:*.tiff=38;5;13:*.png=38;
>      5;13:*.svg=38;5;13:*.svgz=38;5;13:*.mng=38;5;13:*.pcx=38;5;13:*.mov=38;5;13:*.mpg=38;5;13:*.mpeg=38;5;13:*.m2v=38;5;13:*.mkv=38;5;13:*.webm=38;5;13:*.ogm=38;5;13:*.mp4=38;5;13:*.m4v=38;5;13:*.mp4v=38;5;13:*.vob=38;5;13:*.qt=38;5;13:*.nuv=38;5;13:*.wmv=38;5;13:*.asf=38;5;13:*.rm=38;5;13:*.rmvb=38;5;13:*.flc=38;5;13:*.avi=38;5;13:*.fli=38;5;13:*.flv=38;5;13:*.gl=38;5;13:*.dl=38;5;13:*.xcf=38;5;13:*.xwd=38;5;13:*.yuv=38;5;13:*.cgm=38;5;13:*.emf=38;5;13:*.axv=38;5;13:*.anx=38;5;13:*.ogv=38;5;13:*.ogx=38;5;13:*.aac=38;5;45:*.au=38;5;45:*.flac=38;5;45:*.mid=38;5;45:*.midi=38;5;45:*.mka=38;5;45:*.mp3=38;5;45:*.mpc=38;5;45:*.ogg=38;5;45:*.ra=38;5;45:*.wav=38;5;45:*.axa=38;5;45:*.oga=38;5;45:*.spx=38;5;45:*.xspf=38;5;45:
>          MAIL=/var/spool/mail/buildbot
>          OLDPWD=/home/buildbot
>     PATH=/usr/lib64/qt-3.3/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/bin:/sbin:/home/buildbot/.local/bin:/home/buildbot/bin
>     PWD=/media/sf_G_DRIVE/buildbot/centos7-worker/gfortran_centos7_build/build
>          QTDIR=/usr/lib64/qt-3.3
>          QTINC=/usr/lib64/qt-3.3/include
>          QTLIB=/usr/lib64/qt-3.3/lib
>          QT_GRAPHICSSYSTEM_CHECKED=1
>          QT_IM_MODULE=ibus
>     SESSION_MANAGER=local/unix:@/tmp/.ICE-unix/2119,unix/unix:/tmp/.ICE-unix/2119
>          SHELL=/bin/bash
>          SHLVL=2
>          SSH_AGENT_PID=2306
>          SSH_AUTH_SOCK=/run/user/1000/keyring/ssh
>          TERM=xterm-256color
>          USER=buildbot
>          USERNAME=buildbot
>          VTE_VERSION=3803
>          WINDOWID=44049837
>          WINDOWPATH=1
>          XAUTHORITY=/run/gdm/auth-for-buildbot-xH6kQ3/database
>          XDG_CURRENT_DESKTOP=GNOME-Classic:GNOME
>          XDG_MENU_PREFIX=gnome-
>          XDG_RUNTIME_DIR=/run/user/1000
>          XDG_SEAT=seat0
>          XDG_SESSION_DESKTOP=gnome-classic
>          XDG_SESSION_ID=1
>          XDG_VTNR=1
>          XMODIFIERS=@im=ibus
>          _=/usr/bin/buildslave
>         using PTY: False
>        svn, version 1.7.14 (r1542130)
>           compiled Nov 20 2015, 19:25:09
>        (subsequent lines omitted)...
>     ------------------------------------------------------------
>
>     I've tried recreating the buildbot master and both of its buildslaves
>     (deleting the master folder and the buildslaves folders, creating the
>     buildbot and buildslaves, then restoring the buildbot.tacs and
>     master.cfg from backup).  And I also unmounted the /media/sf_G_DRIVE
>     altogether.  All of this seemed to purge the log as expected, but the
>     next build attempt still makes reference to the old location of the
>     CentOS buildslave in /media/sf_G_DRIVE drive.
>
>     Where might the reference to this obsolete location of the
>     buildslave be
>     stored?  The centos7-worker's buildbot.tac file contains no explicit
>     reference to the obsolete location:
>     ------------------------------------------------------------
>
>        import os
>
>        from buildslave.bot import BuildSlave
>        from twisted.application import service
>
>        basedir = '.'
>        rotateLength = 10000000
>        maxRotatedFiles = 10
>
>        # if this is a relocatable tac file, get the directory
>     containing the TAC
>        if basedir == '.':
>            import os.path
>            basedir = os.path.abspath(os.path.dirname(__file__))
>
>        # note: this line is matched against to check that this is a
>     buildslave
>        # directory; do not edit it.
>        application = service.Application('buildslave')
>
>        try:
>          from twisted.python.logfile import LogFile
>          from twisted.python.log import ILogObserver, FileLogObserver
>          logfile = LogFile.fromFullPath(os.path.join(basedir,
>     "twistd.log"),
>     rotateLength=rotateLength,
>     maxRotatedFiles=maxRotatedFiles)
>          application.setComponent(ILogObserver,
>     FileLogObserver(logfile).emit)
>        except ImportError:
>          # probably not yet twisted 8.2.0 and beyond, can't set log yet
>          pass
>
>        buildmaster_host = 'kraken'
>        port = 9989
>        slavename = 'centos7-worker'
>        passwd = 'pass'
>        keepalive = 600
>        usepty = 0
>        umask = None
>        maxdelay = 300
>        allow_shutdown = None
>
>        s = BuildSlave(buildmaster_host, port, slavename, passwd, basedir,
>                       keepalive, usepty, umask=umask, maxdelay=maxdelay,
>                       allow_shutdown=allow_shutdown)
>        s.setServiceParent(application)
>
>
>     Likewise, the master.cfg file creates a single SVN checkout step as
>
>        checkout = SVN(repourl  = 'svn://kraken/trunk/sf_code',
>                       username = "buildbot",
>                       password = "buildbot",
>                       haltOnFailure = True,
>                       )
>     ------------------------------------------------------------
>
>     and this step is used in all builders on both buildslaves (the
>     "centos7-worker" and the "win64-worker"), so I don't see how it could
>     convey the obsolete location to the buildslave.
>
>
>     --
>     Greg Bullock
>     NorthWest Research Associates
>     301 Webster St.
>     Monterey, CA  93940
>     (831) 582-4907
>     greg at nwra.com <mailto:greg at nwra.com>
>
>
>     _______________________________________________
>     users mailing list
>     users at buildbot.net <mailto:users at buildbot.net>
>     https://lists.buildbot.net/mailman/listinfo/users
>
>
>
> _______________________________________________
> users mailing list
> users at buildbot.net
> https://lists.buildbot.net/mailman/listinfo/users

-- 
Regards.
Greg Bullock
NorthWest Research Associates
301 Webster St.
Monterey CA 93940
(831) 582-4907
greg at nwra.com

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.buildbot.net/pipermail/users/attachments/20160616/50ffaffb/attachment.html>


More information about the users mailing list