[Buildbot-commits] [Buildbot] #2424: cloning git repositories over http/https requires usePTY=1
Buildbot
nobody at buildbot.net
Sat Jan 12 16:15:26 UTC 2013
#2424: cloning git repositories over http/https requires usePTY=1
--------------------+-------------------------
Reporter: jhford | Owner:
Type: defect | Status: closed
Priority: minor | Milestone: undecided
Version: 0.8.7p1 | Resolution: worksforme
Keywords: |
--------------------+-------------------------
Changes (by dustin):
* status: new => closed
* resolution: => worksforme
Old description:
> I am trying to clone some Git repositories using the 'repo' tool, invoked
> under a ShellCommand. I've also tested using both string and list
> command of raw git in shellcommand. When I do this on the command line
> or under buildbot with usePTY=1, everything works fine and the clones
> complete. When I run this command under buildbot with usePTY=0, I get
> errors from the CURL library that git uses.
>
> This happens on both OS X and Linux, and there is no reason that I can
> think of for curl needing a pty. Is there documentation on the
> differences between a usePTY environment an a non-usePTY environment?
>
> As a diagnostic, at Dustin's suggestion, I ran the same command on the
> command like with </dev/null and it continued to work from there.
>
> Log from an example error, but this is happening with multiple git
> servers:
>
> (view as text)
> git clone https://git.mozilla.org/releases/gecko.git
> in dir /Volumes/Boot2Gecko/b2g-CI/buildbot/slave/simple-mac/test
> (timeout 1200 secs)
> watching logfiles {}
> argv: git clone https://git.mozilla.org/releases/gecko.git
> environment:
> ARCHFLAGS=-arch i386 -arch x86_64
> Apple_PubSub_Socket_Render=/tmp/launch-Jyxydm/Render
> Apple_Ubiquity_Message=/tmp/launch-toVU3J/Apple_Ubiquity_Message
> CCACHE_COMPRESS=1
> CCACHE_MAX_SIZE=4G
> COLORFGBG=7;0
> COMMAND_MODE=unix2003
> DBUS_LAUNCHD_SESSION_BUS_SOCKET=/tmp/launch-rRn9v2/unix_domain_listener
> DISPLAY=/tmp/launch-frWZnc/org.x:0
> GIT_CURL_VERBOSE=1
> GPG_AGENT_INFO=/Users/jhford/.gnupg/S.gpg-agent:171:1
> HOME=/Users/jhford
> ITERM_PROFILE=Default
> ITERM_SESSION_ID=w0t10p0
> LANG=en_US.UTF-8
> LOGNAME=jhford
> MOZ_LDAP_SHORTUSER=jford
> OLDPWD=/Users/jhford/b2g/b2g-CI/buildbot/master
> PATH=/Volumes/Boot2Gecko/b2g-
> CI/buildbot/bin:/opt/gnu/bin:~/bin:~/sbin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/X11/bin:/usr/local/MacGPG2/bin:/Users/jhford/b2g
> /android-sdk-macosx/platform-tools
> PS1=(buildbot)\h:\w $
> PWD=/Volumes/Boot2Gecko/b2g-CI/buildbot/slave/simple-mac/test
> SHELL=/bin/bash
> SHLVL=1
> SSH_AUTH_SOCK=/tmp/launch-DG8HEW/Listeners
> TERM=xterm-256color
> TERM_PROGRAM=iTerm.app
> TMPDIR=/var/folders/tw/_qcnjxb53wj3yq54wpyl761w0000gn/T/
> USER=jhford
> VERSIONER_PYTHON_PREFER_32_BIT=no
> VERSIONER_PYTHON_VERSION=2.7
> VIRTUAL_ENV=/Volumes/Boot2Gecko/b2g-CI/buildbot
> _=/Volumes/Boot2Gecko/b2g-CI/buildbot/bin/buildslave
> __CF_USER_TEXT_ENCODING=0x1F5:0:0
> using PTY: False
> Cloning into 'gecko'...
> * Couldn't find host git.mozilla.org in the .netrc file; using defaults
> * About to connect() to git.mozilla.org port 443 (#0)
> * Trying 63.245.215.69... * Connected to git.mozilla.org
> (63.245.215.69) port 443 (#0)
> * SSL connection using RC4-SHA
> * Server certificate:
> * subject: serialNumber=ZAUg9vz7QmuprDtV/R1Fd1h--k98QCOX; C=US;
> ST=California; L=Mountain View; O=Mozilla Corporation; OU=IT;
> CN=git.mozilla.org
> * start date: 2012-06-28 01:15:57 GMT
> * expire date: 2014-06-30 21:30:24 GMT
> * subjectAltName: git.mozilla.org matched
> * issuer: C=US; O=GeoTrust, Inc.; CN=GeoTrust SSL CA
> * SSL certificate verify ok.
> > GET /releases/gecko.git/info/refs?service=git-upload-pack HTTP/1.1
> User-Agent: git/1.7.7.5 (Apple Git-26)
> Host: git.mozilla.org
> Accept: */*
> Pragma: no-cache
>
> < HTTP/1.1 200 OK
> < Server: Apache/2.2.15 (Red Hat)
> < X-Backend-Server: git1
> < Cache-Control: no-cache, max-age=0, must-revalidate
> < Content-Type: application/x-git-upload-pack-advertisement
> < Date: Fri, 11 Jan 2013 13:56:57 GMT
> < Expires: Fri, 01 Jan 1980 00:00:00 GMT
> < Pragma: no-cache
> < Transfer-Encoding: chunked
> < Connection: Keep-Alive
> <
> * Connection #0 to host git.mozilla.org left intact
> * Couldn't find host git.mozilla.org in the .netrc file; using defaults
> * About to connect() to git.mozilla.org port 443 (#0)
> * Trying 63.245.215.69... * connected
> * Connected to git.mozilla.org (63.245.215.69) port 443 (#0)
> * SSL re-using session ID
> * SSL connection using RC4-SHA
> * Server certificate:
> * subject: serialNumber=ZAUg9vz7QmuprDtV/R1Fd1h--k98QCOX; C=US;
> ST=California; L=Mountain View; O=Mozilla Corporation; OU=IT;
> CN=git.mozilla.org
> * start date: 2012-06-28 01:15:57 GMT
> * expire date: 2014-06-30 21:30:24 GMT
> * subjectAltName: git.mozilla.org matched
> * issuer: C=US; O=GeoTrust, Inc.; CN=GeoTrust SSL CA
> * SSL certificate verify ok.
> > POST /releases/gecko.git/git-upload-pack HTTP/1.1
> User-Agent: git/1.7.7.5 (Apple Git-26)
> Host: git.mozilla.org
> Accept-Encoding: deflate, gzip
> Content-Type: application/x-git-upload-pack-request
> Accept: application/x-git-upload-pack-result
> Content-Length: 286
>
> < HTTP/1.1 200 OK
> < Server: Apache/2.2.15 (Red Hat)
> < X-Backend-Server: git1
> < Cache-Control: no-cache, max-age=0, must-revalidate
> < Content-Type: application/x-git-upload-pack-result
> < Date: Fri, 11 Jan 2013 13:56:58 GMT
> < Expires: Fri, 01 Jan 1980 00:00:00 GMT
> < Pragma: no-cache
> < Transfer-Encoding: chunked
> < Connection: Keep-Alive
> <
> * transfer closed with outstanding read data remaining
> * Closing connection #0
> error: RPC failed; result=18, HTTP code = 200
> fatal: The remote end hung up unexpectedly
> fatal: protocol error: bad pack header
> program finished with exit code 128
> elapsedTime=44.583997
New description:
I am trying to clone some Git repositories using the 'repo' tool, invoked
under a ShellCommand. I've also tested using both string and list command
of raw git in shellcommand. When I do this on the command line or under
buildbot with usePTY=1, everything works fine and the clones complete.
When I run this command under buildbot with usePTY=0, I get errors from
the CURL library that git uses.
This happens on both OS X and Linux, and there is no reason that I can
think of for curl needing a pty. Is there documentation on the
differences between a usePTY environment an a non-usePTY environment?
As a diagnostic, at Dustin's suggestion, I ran the same command on the
command like with </dev/null and it continued to work from there.
Log from an example error, but this is happening with multiple git
servers:
{{{
(view as text)
git clone https://git.mozilla.org/releases/gecko.git
in dir /Volumes/Boot2Gecko/b2g-CI/buildbot/slave/simple-mac/test (timeout
1200 secs)
watching logfiles {}
argv: git clone https://git.mozilla.org/releases/gecko.git
environment:
ARCHFLAGS=-arch i386 -arch x86_64
Apple_PubSub_Socket_Render=/tmp/launch-Jyxydm/Render
Apple_Ubiquity_Message=/tmp/launch-toVU3J/Apple_Ubiquity_Message
CCACHE_COMPRESS=1
CCACHE_MAX_SIZE=4G
COLORFGBG=7;0
COMMAND_MODE=unix2003
DBUS_LAUNCHD_SESSION_BUS_SOCKET=/tmp/launch-rRn9v2/unix_domain_listener
DISPLAY=/tmp/launch-frWZnc/org.x:0
GIT_CURL_VERBOSE=1
GPG_AGENT_INFO=/Users/jhford/.gnupg/S.gpg-agent:171:1
HOME=/Users/jhford
ITERM_PROFILE=Default
ITERM_SESSION_ID=w0t10p0
LANG=en_US.UTF-8
LOGNAME=jhford
MOZ_LDAP_SHORTUSER=jford
OLDPWD=/Users/jhford/b2g/b2g-CI/buildbot/master
PATH=/Volumes/Boot2Gecko/b2g-
CI/buildbot/bin:/opt/gnu/bin:~/bin:~/sbin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/X11/bin:/usr/local/MacGPG2/bin:/Users/jhford/b2g
/android-sdk-macosx/platform-tools
PS1=(buildbot)\h:\w $
PWD=/Volumes/Boot2Gecko/b2g-CI/buildbot/slave/simple-mac/test
SHELL=/bin/bash
SHLVL=1
SSH_AUTH_SOCK=/tmp/launch-DG8HEW/Listeners
TERM=xterm-256color
TERM_PROGRAM=iTerm.app
TMPDIR=/var/folders/tw/_qcnjxb53wj3yq54wpyl761w0000gn/T/
USER=jhford
VERSIONER_PYTHON_PREFER_32_BIT=no
VERSIONER_PYTHON_VERSION=2.7
VIRTUAL_ENV=/Volumes/Boot2Gecko/b2g-CI/buildbot
_=/Volumes/Boot2Gecko/b2g-CI/buildbot/bin/buildslave
__CF_USER_TEXT_ENCODING=0x1F5:0:0
using PTY: False
Cloning into 'gecko'...
* Couldn't find host git.mozilla.org in the .netrc file; using defaults
* About to connect() to git.mozilla.org port 443 (#0)
* Trying 63.245.215.69... * Connected to git.mozilla.org (63.245.215.69)
port 443 (#0)
* SSL connection using RC4-SHA
* Server certificate:
* subject: serialNumber=ZAUg9vz7QmuprDtV/R1Fd1h--k98QCOX; C=US;
ST=California; L=Mountain View; O=Mozilla Corporation; OU=IT;
CN=git.mozilla.org
* start date: 2012-06-28 01:15:57 GMT
* expire date: 2014-06-30 21:30:24 GMT
* subjectAltName: git.mozilla.org matched
* issuer: C=US; O=GeoTrust, Inc.; CN=GeoTrust SSL CA
* SSL certificate verify ok.
> GET /releases/gecko.git/info/refs?service=git-upload-pack HTTP/1.1
User-Agent: git/1.7.7.5 (Apple Git-26)
Host: git.mozilla.org
Accept: */*
Pragma: no-cache
< HTTP/1.1 200 OK
< Server: Apache/2.2.15 (Red Hat)
< X-Backend-Server: git1
< Cache-Control: no-cache, max-age=0, must-revalidate
< Content-Type: application/x-git-upload-pack-advertisement
< Date: Fri, 11 Jan 2013 13:56:57 GMT
< Expires: Fri, 01 Jan 1980 00:00:00 GMT
< Pragma: no-cache
< Transfer-Encoding: chunked
< Connection: Keep-Alive
<
* Connection #0 to host git.mozilla.org left intact
* Couldn't find host git.mozilla.org in the .netrc file; using defaults
* About to connect() to git.mozilla.org port 443 (#0)
* Trying 63.245.215.69... * connected
* Connected to git.mozilla.org (63.245.215.69) port 443 (#0)
* SSL re-using session ID
* SSL connection using RC4-SHA
* Server certificate:
* subject: serialNumber=ZAUg9vz7QmuprDtV/R1Fd1h--k98QCOX; C=US;
ST=California; L=Mountain View; O=Mozilla Corporation; OU=IT;
CN=git.mozilla.org
* start date: 2012-06-28 01:15:57 GMT
* expire date: 2014-06-30 21:30:24 GMT
* subjectAltName: git.mozilla.org matched
* issuer: C=US; O=GeoTrust, Inc.; CN=GeoTrust SSL CA
* SSL certificate verify ok.
> POST /releases/gecko.git/git-upload-pack HTTP/1.1
User-Agent: git/1.7.7.5 (Apple Git-26)
Host: git.mozilla.org
Accept-Encoding: deflate, gzip
Content-Type: application/x-git-upload-pack-request
Accept: application/x-git-upload-pack-result
Content-Length: 286
< HTTP/1.1 200 OK
< Server: Apache/2.2.15 (Red Hat)
< X-Backend-Server: git1
< Cache-Control: no-cache, max-age=0, must-revalidate
< Content-Type: application/x-git-upload-pack-result
< Date: Fri, 11 Jan 2013 13:56:58 GMT
< Expires: Fri, 01 Jan 1980 00:00:00 GMT
< Pragma: no-cache
< Transfer-Encoding: chunked
< Connection: Keep-Alive
<
* transfer closed with outstanding read data remaining
* Closing connection #0
error: RPC failed; result=18, HTTP code = 200
fatal: The remote end hung up unexpectedly
fatal: protocol error: bad pack header
program finished with exit code 128
elapsedTime=44.583997
}}}
--
Comment:
This is definitely weird, but as far as Buildbot's concerned, there's no
bug here - git and/or curl needs usePty=1. The bug's in Git or Curl.
{{{
dustin at euclid ~ $ python -c 'import os; print os.isatty(0); print
os.isatty(1); print os.isatty(2)' < /dev/null >x 2>&1
dustin at euclid ~ $ cat x
False
False
False
}}}
and running git in that environment:
{{{
dustin at euclid ~/tmp $ GIT_CURL_VERBOSE=1 git clone
http://git.mozilla.org/releases/gecko.git < /dev/null >x 2>&1
dustin at euclid ~/tmp $ echo $?
128
dustin at euclid ~/tmp $ cat x
Cloning into 'gecko'...
* Couldn't find host git.mozilla.org in the .netrc file; using defaults
* About to connect() to git.mozilla.org port 80 (#0)
* Trying 63.245.215.69...
* Connected to git.mozilla.org (63.245.215.69) port 80 (#0)
* Connected to git.mozilla.org (63.245.215.69) port 80 (#0)
> GET /releases/gecko.git/info/refs?service=git-upload-pack HTTP/1.1
User-Agent: git/1.8.0.2
Host: git.mozilla.org
Accept: */*
Accept-Encoding: gzip
Pragma: no-cache
< HTTP/1.1 200 OK
< Server: Apache/2.2.15 (Red Hat)
< X-Backend-Server: git1
< Cache-Control: no-cache, max-age=0, must-revalidate
< Content-Type: application/x-git-upload-pack-advertisement
< Date: Sat, 12 Jan 2013 16:12:45 GMT
< Expires: Fri, 01 Jan 1980 00:00:00 GMT
< Pragma: no-cache
< Transfer-Encoding: chunked
< Connection: Keep-Alive
<
* Connection #0 to host git.mozilla.org left intact
* Couldn't find host git.mozilla.org in the .netrc file; using defaults
* About to connect() to git.mozilla.org port 80 (#0)
* Trying 63.245.215.69...
* connected
* Connected to git.mozilla.org (63.245.215.69) port 80 (#0)
> POST /releases/gecko.git/git-upload-pack HTTP/1.1
User-Agent: git/1.8.0.2
Host: git.mozilla.org
Accept-Encoding: gzip
Content-Type: application/x-git-upload-pack-request
Accept: application/x-git-upload-pack-result
Content-Length: 286
* upload completely sent off: 286 out of 286 bytes
< HTTP/1.1 200 OK
< Server: Apache/2.2.15 (Red Hat)
< X-Backend-Server: git1
< Cache-Control: no-cache, max-age=0, must-revalidate
< Content-Type: application/x-git-upload-pack-result
< Date: Sat, 12 Jan 2013 16:12:46 GMT
< Expires: Fri, 01 Jan 1980 00:00:00 GMT
< Pragma: no-cache
< Transfer-Encoding: chunked
< Connection: Keep-Alive
<
* transfer closed with outstanding read data remaining
* Closing connection #0
error: RPC failed; result=18, HTTP code = 200
fatal: The remote end hung up unexpectedly
fatal: protocol error: bad pack header
}}}
This would be interesting to debug as a git/curl bug, but not here.
--
Ticket URL: <http://trac.buildbot.net/ticket/2424#comment:1>
Buildbot <http://buildbot.net/>
Buildbot: build/test automation
More information about the Commits
mailing list