[Buildbot-devel] Removing the "lone pipe exception"

[Dustin J. Mitchell]

Well, we do use Twisted's `spawnProcess`, but when given a string we
want to replicate the behavior on POSIX, which is to pass it to sh -c
'..'.  Although I see (in `slave/buildslave/runprocess.py`) that we're
still using COMSPEC even when given a list.

Maybe a good way to start to get into this is to assemble a set of
desired behaviors (run an EXE, and pipe output between two EXEs might
be two of them), and then a set of existing configurations so we know
if and when we're breaking backward compatibliity (something like
ShellCommand('echo "hello" | NUL')?).  That would help to define the
problem we're trying to solve.  Once that's done, if switching from
`spawnProcess` to `subprocess.Popen()` gets us more behaviors with
minimal impact to compatibility, great!  If dumping the command into
$temp.bat and running that batch file is better, then let's do that.

It seems like there's a solid group of people with enough Windows
experience to hammer this out -- I'm just not one of them :)

Dustin

On Tue, Sep 23, 2014 at 5:02 PM, Vasily <vasslitvinov at pisem.net> wrote:
> Is there any reason why don't we just use Python's own subprocess.Popen()
> and let it do all the quoting? It has always puzzled me...
>
> Thanks,
> Vasily
>
> 24 сент. 2014 г. 0:30 пользователь "Dustin J. Mitchell" <dustin at v.igoro.us>
> написал:
>
>> Couldn't you pass "^|" explicitly?
>>
>> My understanding of Windows quoting issues is minimal -- I recall
>> being referred to a very long document about CMD.EXE explaining how
>> quoting and unquoting was performed partially by CMD.EXE and partially
>> by the executable being started, and that the behavior differed
>> between different OS versions.  I kinda gave up at that point.
>>
>> All of which is to say, I'd like to have a consistent deterministic
>> quoting system that makes sense to those familiar with Windows and
>> that is as simple as possible.  I don't particularly mind if it's
>> consistent with other OSes.  If reverting this change is a part of
>> that new quoting system, great, but I'm a little hesitant to bless
>> reverting a change that made great sense to someone else.
>>
>> Dustin
>>
>> On Tue, Sep 23, 2014 at 3:51 PM, Роман Донченко <dpb at corrigendum.ru>
>> wrote:
>> > Hello all,
>> >
>> > On Windows, if ShellCommand is used with a list of arguments, an
>> > argument
>> > consisting of a single pipe character will not be escaped, as per the
>> > documentation:
>> >
>> >> On the topic of shell metacharacters, note that in DOS the pipe
>> >> character (|) is conditionally escaped (to ^|) when it occurs inside a
>> >> more complex string in a list of strings. It remains unescaped when it
>> >> occurs as part of a single string or as a lone pipe in a list of
>> >> strings.
>> >
>> > I'm not a fan of this behaviour, since it:
>> >
>> > 1) is inconsistent across OSes;
>> > 2) means that you can't pass an actual solitary pipe as an argument.
>> >
>> > So I'd like to remove it - that is, make it so that a pipe is always
>> > escaped no matter what. It is a backwards-incompatible change, but I
>> > think
>> > for Nine it should be okay. Still, I'd like to receive developers'
>> > blessing (or at least a lack of cursing) before I make the change.
>> >
>> > So what do you thinK?
>> >
>> > Roman.
>> >
>> >
>> >
>> > ------------------------------------------------------------------------------
>> > Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer
>> > Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports
>> > Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper
>> > Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer
>> >
>> > http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg.clktrk
>> > _______________________________________________
>> > Buildbot-devel mailing list
>> > Buildbot-devel at lists.sourceforge.net
>> > https://lists.sourceforge.net/lists/listinfo/buildbot-devel
>>
>>
>> ------------------------------------------------------------------------------
>> Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer
>> Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports
>> Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper
>> Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer
>>
>> http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg.clktrk
>> _______________________________________________
>> Buildbot-devel mailing list
>> Buildbot-devel at lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/buildbot-devel