[Buildbot-commits] [Buildbot] #2148: buildbot.schedulers.trysched.BadJobfile: unable to parse netstrings
Buildbot
nobody at buildbot.net
Thu Nov 17 07:09:51 UTC 2011
#2148: buildbot.schedulers.trysched.BadJobfile: unable to parse netstrings
----------------------+--------------------
Reporter: mlakewood | Owner:
Type: defect | Status: new
Priority: major | Milestone: 0.8.6
Version: 0.8.4p2 | Resolution:
Keywords: try |
----------------------+--------------------
Changes (by dustin):
* keywords: NetString Buildbot Try => try
* milestone: undecided => 0.8.6
Old description:
> Hi all,
>
> Im using buildbot try, using jobdir and ssh to submit builds to buildbot.
> Sometimes I get the following error/traceback in the twistd.log of the
> buildbot server and the job isnt run
>
> 11-11-16 16:23:21+0800 [Broker,0,127.0.0.1] slave 'slave' attaching from
> IPv4Address(TCP, '127.0.0.1', 38491)
> 2011-11-16 16:23:21+0800 [Broker,0,127.0.0.1] Starting buildslave
> keepalive timer for 'slave'
> 2011-11-16 16:23:21+0800 [Broker,0,127.0.0.1] Got slaveinfo from 'slave'
> 2011-11-16 16:23:21+0800 [Broker,0,127.0.0.1] bot attached
> 2011-11-16 16:23:21+0800 [Broker,0,127.0.0.1] Buildslave slave attached
> to sws-build_try
> 2011-11-16 16:23:21+0800 [Broker,0,127.0.0.1] Buildslave slave attached
> to sws-build
> 2011-11-16 16:23:50+0800 [-] <buildbot.schedulers.trysched.Try_Jobdir
> instance at 0xa6d286c> reports a bad jobfile in
> 1321431829-d57b4f44f31d2eea16e0a88732cdbce6
> 2011-11-16 16:23:50+0800 [-] Unhandled Error
> Traceback (most recent call last):
> File "/usr/local/lib/python2.6/dist-
> packages/Twisted-11.0.0-py2.6-linux-i686.egg/twisted/internet/defer.py",
> line 964, in unwindGenerator
> return _deferGenerator(f(*args, **kwargs), Deferred())
> File "/usr/local/lib/python2.6/dist-
> packages/Twisted-11.0.0-py2.6-linux-i686.egg/twisted/internet/defer.py",
> line 866, in _deferGenerator
> result = g.next()
> File "/usr/local/lib/python2.6/dist-
> packages/buildbot/util/maildir.py", line 124, in poll
> wfd = defer.waitForDeferred(self.messageReceived(n))
> File "/usr/local/lib/python2.6/dist-
> packages/buildbot/schedulers/trysched.py", line 66, in messageReceived
> return self.parent.handleJobFile(filename, f)
> --- <exception caught here> ---
> File "/usr/local/lib/python2.6/dist-
> packages/buildbot/schedulers/trysched.py", line 148, in handleJobFile
> parsed_job = self.parseJob(f)
> File "/usr/local/lib/python2.6/dist-
> packages/buildbot/schedulers/trysched.py", line 100, in parseJob
> raise BadJobfile("unable to parse netstrings")
> buildbot.schedulers.trysched.BadJobfile: unable to parse
> netstrings
>
> I've sent the file and our config to djmitche that was helping out on the
> IRC channel. Our setup is
>
> Client Submitting job
>
> Ubuntu 10.04,
> python 2.6.5
> Twistd 10
> Buildbot 0.8.5
>
> Server
> Ubuntu 10.04
> python 2.6.5
> Twisted 11
> Buildbot 0.8.4p2
>
> Cheers
> Mark
New description:
Hi all,
Im using buildbot try, using jobdir and ssh to submit builds to buildbot.
Sometimes I get the following error/traceback in the twistd.log of the
buildbot server and the job isnt run
{{{
11-11-16 16:23:21+0800 [Broker,0,127.0.0.1] slave 'slave' attaching from
IPv4Address(TCP, '127.0.0.1', 38491)
2011-11-16 16:23:21+0800 [Broker,0,127.0.0.1] Starting buildslave
keepalive timer for 'slave'
2011-11-16 16:23:21+0800 [Broker,0,127.0.0.1] Got slaveinfo from 'slave'
2011-11-16 16:23:21+0800 [Broker,0,127.0.0.1] bot attached
2011-11-16 16:23:21+0800 [Broker,0,127.0.0.1] Buildslave slave attached to
sws-build_try
2011-11-16 16:23:21+0800 [Broker,0,127.0.0.1] Buildslave slave attached to
sws-build
2011-11-16 16:23:50+0800 [-] <buildbot.schedulers.trysched.Try_Jobdir
instance at 0xa6d286c> reports a bad jobfile in
1321431829-d57b4f44f31d2eea16e0a88732cdbce6
2011-11-16 16:23:50+0800 [-] Unhandled Error
Traceback (most recent call last):
File "/usr/local/lib/python2.6/dist-
packages/Twisted-11.0.0-py2.6-linux-i686.egg/twisted/internet/defer.py",
line 964, in unwindGenerator
return _deferGenerator(f(*args, **kwargs), Deferred())
File "/usr/local/lib/python2.6/dist-
packages/Twisted-11.0.0-py2.6-linux-i686.egg/twisted/internet/defer.py",
line 866, in _deferGenerator
result = g.next()
File "/usr/local/lib/python2.6/dist-
packages/buildbot/util/maildir.py", line 124, in poll
wfd = defer.waitForDeferred(self.messageReceived(n))
File "/usr/local/lib/python2.6/dist-
packages/buildbot/schedulers/trysched.py", line 66, in messageReceived
return self.parent.handleJobFile(filename, f)
--- <exception caught here> ---
File "/usr/local/lib/python2.6/dist-
packages/buildbot/schedulers/trysched.py", line 148, in handleJobFile
parsed_job = self.parseJob(f)
File "/usr/local/lib/python2.6/dist-
packages/buildbot/schedulers/trysched.py", line 100, in parseJob
raise BadJobfile("unable to parse netstrings")
buildbot.schedulers.trysched.BadJobfile: unable to parse
netstrings
}}}
I've sent the file and our config to djmitche that was helping out on the
IRC channel. Our setup is
Client Submitting job
{{{
Ubuntu 10.04,
python 2.6.5
Twistd 10
Buildbot 0.8.5
}}}
Server
{{{
Ubuntu 10.04
python 2.6.5
Twisted 11
Buildbot 0.8.4p2
}}}
Cheers
Mark
--
Comment:
So I had a look at the file. Two odd things. First, it has a header of
spaces in this format (* for space):
{{{
*********************************************************************
*********************************************************************
*********************************************************************
*********************************************
}}}
which may have been an artifact of the email attachment -- who knows.
Stripping that, what looks like a proper netstring begins:
{{{
1:2,17:1321431405-417459,0:,3:283,1:0,106896:Index:
...
}}}
that portion of the netstring is, in fact, 106896 bytes long, and finishes
with a few more netstring components of the proper length.
Having a look at the source, though:
{{{
#!python
MAX_LENGTH = 99999
}}}
(twisted/protocols/basic.py:115)
so I think it's a matter of netstring length, as you suggested in IRC.
Probably the easiest thing to do is simply increase that in your
master.cfg:
{{{
#!python
from twisted.protocols.basic import NetstringReceiver
NetstringReceiver.MAX_LENGTH = 1000000
}}}
I'm not sure where best to include this in the documentation. 100KiB
seems a bit small for a maximum allowed patch size in the try scheduler,
but having no limits is problematic for potential memory consumption.
--
Ticket URL: <http://trac.buildbot.net/ticket/2148#comment:1>
Buildbot <http://buildbot.net/>
Buildbot: build/test automation
More information about the Commits
mailing list