[users at bb.net] buildbot retains memory of an earlier location of a buildslave?
Greg Bullock
greg at nwra.com
Fri Jun 17 17:03:54 UTC 2016
The step page indeed shows the start time and end time the step, but
clicking the associated link under the Logs section displays content
that itself includes no time stamp.
For my current purposes, I'd like that log content to also report the
date & time.
Some anomaly on my system causes the buildbot to retain prior memory and
ultimately insert into the log a clue as to what it remembers. I've
been thinking that its memory is confined to just the workdir -- related
to location of the buildslave. But so far I can't rule out that it's
the entire log content getting pulled from outdated memory.
On 6/16/2016 10:45 AM, Pierre Tardy wrote:
>
> There is a start time and end time for each step that you can see in
> the step page
>
>
> Le jeu. 16 juin 2016 19:19, Greg Bullock <greg at nwra.com
> <mailto:greg at nwra.com>> a écrit :
>
> It dawns on me that the problem may not be that the buildbot
> somehow retains a memory of the earlier location of a buildslave.
> It could instead be that the entire log that displays is actually
> from an earlier (and failed) build attempt.
>
> Is there a way to have buildbot insert a datetime stamp into the
> log for a given step?
>
>
>
> On 6/14/2016 3:01 PM, Greg Bullock wrote:
>> I'm using version 0.8.12.
>>
>> I have indeed rebooted the master. I've also deleted the CentOS
>> virtual machine (the slave) and created a new one from a fresh
>> install of the system. No luck.
>>
>> I have not tried both of these measures in the same test, I
>> wouldn't think that could make the difference.
>>
>> I will look into purging the sun cache, from the log, it seems
>> that the buildslave is already reporting using the obsolete (and
>> nonexistent) working directory before it calls svn.
>>
>> Greg Bullock
>> NorthWest Research Associates
>> 301 Webster St.
>> Monterey, CA 93940
>> (831) 582-4907 <tel:%28831%29%20582-4907>
>> greg at nwra.com <mailto:greg at nwra.com>
>>
>> -------- Original message --------
>> From: Pierre Tardy <tardyp at gmail.com> <mailto:tardyp at gmail.com>
>> Date: 6/14/16 1:12 PM (GMT-08:00)
>> To: Greg Bullock <greg at nwra.com> <mailto:greg at nwra.com>,
>> users at buildbot.net <mailto:users at buildbot.net>
>> Subject: Re: [users at bb.net <mailto:users at bb.net>] buildbot
>> retains memory of an earlier location of a buildslave?
>>
>> Hi,
>> Looks indeed like an interresting problem. Did you try to reboot
>> the master?
>>
>> The master might indeed store the workdir, but I would expect it
>> to be reset when the work is reconnecting.
>>
>> Which version of buildbot is that?
>>
>> Also thing to check is svn. could svn cache something?
>>
>> Regards
>> Pierre
>>
>> Le mar. 14 juin 2016 à 21:04, Greg Bullock <greg at nwra.com
>> <mailto:greg at nwra.com>> a écrit :
>>
>> Where might the buildbot retain its memory of an earlier
>> location of a
>> buildslave? I've moved a buildslave to a different drive,
>> but the log
>> from the new build tests shows the steps (e.g., the SVN,
>> ShellCommand,
>> and Compile steps) all running from the original drive, not
>> the new drive.
>>
>> My buildbot runs on Windows 7 64-bit, and this particular
>> buildslave
>> (called "centos7-worker") runs on a virtual machine on CentOS
>> 7 (linux)
>> hosted on the same Windows machine with the buildbot master.
>>
>> Originally, the buildslave's files were on a shared drive
>> (which is also
>> a Windows share shared from the Windows host), in
>> /media/sf_G_DRIVE/buildbot/centos7-worker, but the SVN
>> checkout step
>> would successfully checkout only about half the project, then
>> give an error
>>
>> ------------------------------------------------------------
>> ... (previous lines omitted)
>> svn: E200030: disk I/O error, executing statement 'RELEASE
>> s10246'
>> svn: E200030: sqlite: disk I/O error
>> svn: E200030: sqlite: disk I/O error
>> svn: E200030: no such savepoint: s10247, executing statement
>> 'RELEASE s10247'
>> svn: E200030: no such savepoint: s10247, executing statement
>> 'ROLLBACK TO s10247'
>> program finished with exit code 1
>> ------------------------------------------------------------
>>
>> After a Google search turned up several mentions of conflicts
>> between
>> Windows shared drive and sqlite (see, for example,
>> http://stackoverflow.com/questions/18782941/tortoisesvn-checkout-sqlite-disk-i-o-error-s10),
>> I moved the buildslave files to a local drive within the virtual
>> machine, with the path /home/buildbot/Documents/centos7-worker.
>>
>> At this point, I expect the log to have references only to
>> the new
>> /home/buildbot location, with no more references to the older
>> /media/sf_G_DRIVE location. But instead, the log still shows
>> steps
>> running "in dir
>> /media/sf_G_DRIVE/buildbot/centos7-worker/...". The top
>> of the log from the SVN checkout step shows an example:
>>
>> ------------------------------------------------------------
>> svn --version
>> in dir
>> /media/sf_G_DRIVE/buildbot/centos7-worker/gfortran_centos7_build/build
>> (timeout 1200 secs)
>> watching logfiles {}
>> argv: ['svn', '--version']
>> environment:
>> DBUS_SESSION_BUS_ADDRESS=unix:abstract=/tmp/dbus-ebvEqaI7D1,guid=b74561f6a34d14afe5191e515759b435
>> DESKTOP_SESSION=gnome-classic
>> DISPLAY=:0
>> GDMSESSION=gnome-classic
>> GDM_LANG=en_US.UTF-8
>> GJS_DEBUG_OUTPUT=stderr
>> GJS_DEBUG_TOPICS=JS ERROR;JS LOG
>> GNOME_DESKTOP_SESSION_ID=this-is-deprecated
>> GNOME_SHELL_SESSION_MODE=classic
>> GPG_AGENT_INFO=/run/user/1000/keyring/gpg:0:1
>> HISTCONTROL=ignoredups
>> HISTSIZE=1000
>> HOME=/home/buildbot
>> HOSTNAME=localhost.localdomain
>> IMSETTINGS_INTEGRATE_DESKTOP=yes
>> IMSETTINGS_MODULE=none
>> LANG=en_US.UTF-8
>> LESSOPEN=||/usr/bin/lesspipe.sh %s
>> LOGNAME=buildbot
>> LS_COLORS=rs=0:di=38;5;27:ln=38;5;51:mh=44;38;5;15:pi=40;38;5;11:so=38;5;13:do=38;5;5:bd=48;5;232;38;5;11:cd=48;5;232;38;5;3:or=48;5;232;38;5;9:mi=05;48;5;232;38;5;15:su=48;5;196;38;5;15:sg=48;5;11;38;5;16:ca=48;5;196;38;5;226:tw=48;5;10;38;5;16:ow=48;5;10;38;5;21:st=48;5;21;38;5;15:ex=38;5;34:*.tar=38;5;9:*.tgz=38;5;9:*.arc=38;5;9:*.arj=38;5;9:*.taz=38;5;9:*.lha=38;5;9:*.lz4=38;5;9:*.lzh=38;5;9:*.lzma=38;5;9:*.tlz=38;5;9:*.txz=38;5;9:*.tzo=38;5;9:*.t7z=38;5;9:*.zip=38;5;9:*.z=38;5;9:*.Z=38;5;9:*.dz=38;5;9:*.gz=38;5;9:*.lrz=38;5;9:*.lz=38;5;9:*.lzo=38;5;9:*.xz=38;5;9:*.bz2=38;5;9:*.bz=38;5;9:*.tbz=38;5;9:*.tbz2=38;5;9:*.tz=38;5;9:*.deb=38;5;9:*.rpm=38;5;9:*.jar=38;5;9:*.war=38;5;9:*.ear=38;5;9:*.sar=38;5;9:*.rar=38;5;9:*.alz=38;5;9:*.ace=38;5;9:*.zoo=38;5;9:*.cpio=38;5;9:*.7z=38;5;9:*.rz=38;5;9:*.cab=38;5;9:*.jpg=38;5;13:*.jpeg=38;5;13:*.gif=38;5;13:*.bmp=38;5;13:*.pbm=38;5;13:*.pgm=38;5;13:*.ppm=38;5;13:*.tga=38;5;13:*.xbm=38;5;13:*.xpm=38;5;13:*.tif=38;5;13:*.tiff=38;5;13:*.png=38;
>> 5;13:*.svg=38;5;13:*.svgz=38;5;13:*.mng=38;5;13:*.pcx=38;5;13:*.mov=38;5;13:*.mpg=38;5;13:*.mpeg=38;5;13:*.m2v=38;5;13:*.mkv=38;5;13:*.webm=38;5;13:*.ogm=38;5;13:*.mp4=38;5;13:*.m4v=38;5;13:*.mp4v=38;5;13:*.vob=38;5;13:*.qt=38;5;13:*.nuv=38;5;13:*.wmv=38;5;13:*.asf=38;5;13:*.rm=38;5;13:*.rmvb=38;5;13:*.flc=38;5;13:*.avi=38;5;13:*.fli=38;5;13:*.flv=38;5;13:*.gl=38;5;13:*.dl=38;5;13:*.xcf=38;5;13:*.xwd=38;5;13:*.yuv=38;5;13:*.cgm=38;5;13:*.emf=38;5;13:*.axv=38;5;13:*.anx=38;5;13:*.ogv=38;5;13:*.ogx=38;5;13:*.aac=38;5;45:*.au=38;5;45:*.flac=38;5;45:*.mid=38;5;45:*.midi=38;5;45:*.mka=38;5;45:*.mp3=38;5;45:*.mpc=38;5;45:*.ogg=38;5;45:*.ra=38;5;45:*.wav=38;5;45:*.axa=38;5;45:*.oga=38;5;45:*.spx=38;5;45:*.xspf=38;5;45:
>> MAIL=/var/spool/mail/buildbot
>> OLDPWD=/home/buildbot
>> PATH=/usr/lib64/qt-3.3/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/bin:/sbin:/home/buildbot/.local/bin:/home/buildbot/bin
>> PWD=/media/sf_G_DRIVE/buildbot/centos7-worker/gfortran_centos7_build/build
>> QTDIR=/usr/lib64/qt-3.3
>> QTINC=/usr/lib64/qt-3.3/include
>> QTLIB=/usr/lib64/qt-3.3/lib
>> QT_GRAPHICSSYSTEM_CHECKED=1
>> QT_IM_MODULE=ibus
>> SESSION_MANAGER=local/unix:@/tmp/.ICE-unix/2119,unix/unix:/tmp/.ICE-unix/2119
>> <mailto:SESSION_MANAGER=local/unix:@/tmp/.ICE-unix/2119,unix/unix:/tmp/.ICE-unix/2119>
>> SHELL=/bin/bash
>> SHLVL=2
>> SSH_AGENT_PID=2306
>> SSH_AUTH_SOCK=/run/user/1000/keyring/ssh
>> TERM=xterm-256color
>> USER=buildbot
>> USERNAME=buildbot
>> VTE_VERSION=3803
>> WINDOWID=44049837
>> WINDOWPATH=1
>> XAUTHORITY=/run/gdm/auth-for-buildbot-xH6kQ3/database
>> XDG_CURRENT_DESKTOP=GNOME-Classic:GNOME
>> XDG_MENU_PREFIX=gnome-
>> XDG_RUNTIME_DIR=/run/user/1000
>> XDG_SEAT=seat0
>> XDG_SESSION_DESKTOP=gnome-classic
>> XDG_SESSION_ID=1
>> XDG_VTNR=1
>> XMODIFIERS=@im=ibus
>> _=/usr/bin/buildslave
>> using PTY: False
>> svn, version 1.7.14 (r1542130)
>> compiled Nov 20 2015, 19:25:09
>> (subsequent lines omitted)...
>> ------------------------------------------------------------
>>
>> I've tried recreating the buildbot master and both of its
>> buildslaves
>> (deleting the master folder and the buildslaves folders,
>> creating the
>> buildbot and buildslaves, then restoring the buildbot.tacs and
>> master.cfg from backup). And I also unmounted the
>> /media/sf_G_DRIVE
>> altogether. All of this seemed to purge the log as expected,
>> but the
>> next build attempt still makes reference to the old location
>> of the
>> CentOS buildslave in /media/sf_G_DRIVE drive.
>>
>> Where might the reference to this obsolete location of the
>> buildslave be
>> stored? The centos7-worker's buildbot.tac file contains no
>> explicit
>> reference to the obsolete location:
>> ------------------------------------------------------------
>>
>> import os
>>
>> from buildslave.bot import BuildSlave
>> from twisted.application import service
>>
>> basedir = '.'
>> rotateLength = 10000000
>> maxRotatedFiles = 10
>>
>> # if this is a relocatable tac file, get the directory
>> containing the TAC
>> if basedir == '.':
>> import os.path
>> basedir = os.path.abspath(os.path.dirname(__file__))
>>
>> # note: this line is matched against to check that this is
>> a buildslave
>> # directory; do not edit it.
>> application = service.Application('buildslave')
>>
>> try:
>> from twisted.python.logfile import LogFile
>> from twisted.python.log import ILogObserver, FileLogObserver
>> logfile = LogFile.fromFullPath(os.path.join(basedir,
>> "twistd.log"),
>> rotateLength=rotateLength,
>> maxRotatedFiles=maxRotatedFiles)
>> application.setComponent(ILogObserver,
>> FileLogObserver(logfile).emit)
>> except ImportError:
>> # probably not yet twisted 8.2.0 and beyond, can't set
>> log yet
>> pass
>>
>> buildmaster_host = 'kraken'
>> port = 9989
>> slavename = 'centos7-worker'
>> passwd = 'pass'
>> keepalive = 600
>> usepty = 0
>> umask = None
>> maxdelay = 300
>> allow_shutdown = None
>>
>> s = BuildSlave(buildmaster_host, port, slavename, passwd,
>> basedir,
>> keepalive, usepty, umask=umask,
>> maxdelay=maxdelay,
>> allow_shutdown=allow_shutdown)
>> s.setServiceParent(application)
>>
>>
>> Likewise, the master.cfg file creates a single SVN checkout
>> step as
>>
>> checkout = SVN(repourl = 'svn://kraken/trunk/sf_code',
>> username = "buildbot",
>> password = "buildbot",
>> haltOnFailure = True,
>> )
>> ------------------------------------------------------------
>>
>> and this step is used in all builders on both buildslaves (the
>> "centos7-worker" and the "win64-worker"), so I don't see how
>> it could
>> convey the obsolete location to the buildslave.
>>
>>
>> --
>> Greg Bullock
>> NorthWest Research Associates
>> 301 Webster St.
>> Monterey, CA 93940
>> (831) 582-4907
>> greg at nwra.com <mailto:greg at nwra.com>
>>
>>
>> _______________________________________________
>> users mailing list
>> users at buildbot.net <mailto:users at buildbot.net>
>> https://lists.buildbot.net/mailman/listinfo/users
>>
>>
>>
>> _______________________________________________
>> users mailing list
>> users at buildbot.net <mailto:users at buildbot.net>
>> https://lists.buildbot.net/mailman/listinfo/users
>
> --
> Regards.
> Greg Bullock
> NorthWest Research Associates
> 301 Webster St.
> Monterey CA 93940
> (831) 582-4907
> greg at nwra.com <mailto:greg at nwra.com>
>
> _______________________________________________
> users mailing list
> users at buildbot.net <mailto:users at buildbot.net>
> https://lists.buildbot.net/mailman/listinfo/users
>
--
Greg Bullock
NorthWest Research Associates
301 Webster St.
Monterey, CA 93940
(831) 582-4907
greg at nwra.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.buildbot.net/pipermail/users/attachments/20160617/1411bae6/attachment.html>
More information about the users
mailing list