[Buildbot-devel] SVN mode copy failed

[Jean-Paul Calderone]

On Tue, 15 Jul 2008 11:54:18 -0700, Brian Warner <warner-buildbot at lothar.com> wrote:
>> >But I get "program finished with exit code -1" in the waterfall. (I
>> >don't provide the whole report, nothing else is revelant) in twisted
>> >log
>>
>> I have been seeing intermittent failures similar to this recently,
>> only on 64bit Linux slaves.  It seems to happen less than 10% of the
>> time (eyeball figure).  If I rebuild, it typically works.  Are you
>> seeing this problem for all builds, or just sometimes?
>
>I have seen this more frequently as of late, and I believe that it has to do
>with the use of a PTY on the buildslave. Try adding --usepty=0 when you
>create the buildslave (or edit the slave's buildbot.tac file to change the
>'usepty = 1' line to read 'usepty = 0').
>
>The buildbot closes stdin of the child process after writes any data that
>needs to be written to stdin (such as the body of a patch that needs to be
>applied). Very few steps actually use data on stdin, so usually the pipe gets
>closed right after process startup. If a PTY is in use, this may cause the
>child process to receive a SIGHUP shortly after startup (but after a random
>delay influenced by the kernel scheduler). Many programs don't catch SIGHUP
>and it causes them to quit. If the SIGHUP arrives late, the program may be
>finished already and it isn't a problem. I suspect that something about
>recent linux kernels or some change in glibc has increased the frequency of
>these sorts of failures.
>
>At least, that's my hunch.. I haven't really been able to prove it. I've
>fussed a bit with not closing stdin, but I don't remember being satisfied
>with the results. I believe that after setting usepty=0 in my own
>buildslaves, the problem went away. I'm considering making usepty=0 the
>default for the next release.

Sounds like a pretty good hunch.  Something I can try to verify, at least.

As for the usepty option, this is something that's bothered me for a while.
I'd much prefer this to be a per-step server-side configuration option than
a per-slave slave-side configuration option.  This would make this problem
easier to fix - `cp´ and other such things could be made to not use ptys,
which should fix them if this is indeed the cause of these problems, and
other commands which depend on the use or absense of a pty could be
controlled more easily and with only server-side configuration this simplies
the slave admin's job.

Jean-Paul