[Buildbot] #3557: add option to join stdout and stderr for ShellCommand

[Buildbot trac]

#3557: add option to join stdout and stderr for ShellCommand
------------------------+-----------------------
Reporter:  kparal       |      Owner:
    Type:  enhancement  |     Status:  new
Priority:  major        |  Milestone:  undecided
 Version:  master       |   Keywords:
------------------------+-----------------------
 Current implementation of `ShellCommand` splits stdout and stderr. That
 has a benefit of being able to color streams, and the drawback of not
 keeping exact chronological ordering (which seems impossible to achieve
 with split streams, according to my research).

 In our project, we value chronological ordering more than stderr coloring.
 That's why we wanted to keep stdout and stderr in a single stream (as bash
 would do by default). We tried to use `usePTY=True` for that, and it
 works, however [http://docs.buildbot.net/latest/manual/cfg-buildsteps.html
 #using-shellcommands your documentation] says:
 > Using a pseudo-terminal brings lots of compatibility problems
 and indeed, we encountered
 [https://phab.qadevel.cloud.fedoraproject.org/T799 a weird issue involving
 doubled newlines].

 We don't really want to deal with weird PTY issues, and honestly using PTY
 is just a workaround for BuildBot splitting streams by default without any
 option to disable it. It's an overkill solution.

 Another solution is to append `2>&1` to the executed command, but that
 means we need to run the command through shell (instead of execute it
 directly) which brings a lot of headaches with shell escaping and
 security. Again, a lot of unnecessary trouble just to work around a
 missing feature.

 You already have `want_stdout` and `want_stderr` options, so you offer
 some configuration in this reagard. Please implement a new option named
 e.g. `merge_stderr` or `merge_streams` which would '''not''' split streams
 when starting the process. It would essentially do the same as
 {{{
 subprocess.call(args, stderr=subprocess.STDOUT)
 }}}

 That's all we need. It does not split streams (nor color them, obviously),
 it keeps chronological ordering, it does not need to allocate PTY with its
 compatibility issues, it does not bring headaches with shell escaping, and
 it should be hopefully very simple to implement.

 Thank you.

--
Ticket URL: <http://trac.buildbot.net/ticket/3557>
Buildbot <http://buildbot.net/>
Buildbot: build/test automation