[Buildbot] #3384: UnicodeDecodeError upon finishing BuildStep run

[Buildbot trac]

#3384: UnicodeDecodeError upon finishing BuildStep run
-----------------------+-----------------------
Reporter:  jollyroger  |      Owner:
    Type:  defect      |     Status:  new
Priority:  major       |  Milestone:  undecided
 Version:  0.8.12      |   Keywords:  buildstep
-----------------------+-----------------------
 Running `npm install` inside a build step may cause a sometimes weird
 output appear like this:

 {{{
 ⚠ pre-build test failed, compiling from source...
 ✔ pre-build test passed successfully!
 ✔ pre-build test passed successfully!

 > optipng-bin at 1.0.1 postinstall
 /home/broq/buildbot/karkinos/rpmbuild/build/BUILD/broq-0.5.2.4-1-g2cf5752/build/www/node_modules
 /grunt-contrib-imagemin/node_modules/imagemin/node_modules/imagemin-
 optipng/node_modules/optipng-bin
 > node lib/install.js

 ✔ pngquant built successfully!
 ⚠ pre-build test failed, compiling from source...
 ✔ optipng built successfully!
 }}}

 While this output was absolutely ok for 0.8.9, it causes buildstep to fail
 with an exception on the first character for 0.8.12:

 {{{
 2015-11-16 20:37:46+0000 [-] BuildStep.failed; traceback follows
         Traceback (most recent call last):
           File "/usr/lib/python2.7/dist-
 packages/twisted/internet/defer.py", line 1101, in gotResult
             _inlineCallbacks(r, g, deferred)
           File "/usr/lib/python2.7/dist-
 packages/twisted/internet/defer.py", line 1088, in _inlineCallbacks
             deferred.callback(e.value)
           File "/usr/lib/python2.7/dist-
 packages/twisted/internet/defer.py", line 368, in callback
             self._startRunCallbacks(result)
           File "/usr/lib/python2.7/dist-
 packages/twisted/internet/defer.py", line 464, in _startRunCallbacks
             self._runCallbacks()
         --- <exception caught here> ---
           File "/usr/lib/python2.7/dist-
 packages/twisted/internet/defer.py", line 551, in _runCallbacks
             current.result = callback(current.result, *args, **kw)
           File "/usr/lib/python2.7/dist-
 packages/buildbot/process/buildstep.py", line 650, in <lambda>
             d.addCallback(lambda res: self.commandComplete(cmd))
           File "/usr/lib/python2.7/dist-packages/buildbot/steps/shell.py",
 line 337, in commandComplete
             self.observer.getStdout(),
           File "/usr/lib/python2.7/dist-
 packages/buildbot/process/logobserver.py", line 127, in getStdout
             return self._get(self.stdout)
           File "/usr/lib/python2.7/dist-
 packages/buildbot/process/logobserver.py", line 119, in _get
             result = u''.join(chunks)
         exceptions.UnicodeDecodeError: 'ascii' codec can't decode byte
 0xe2 in position 512: ordinal not in range(128)

 2015-11-16 20:37:46+0000 [-] addCompleteLog(err.text)
 2015-11-16 20:37:46+0000 [-] addHTMLLog(err.html)
 2015-11-16 20:37:46+0000 [-] releaseLocks(<broqbot.steps.RpmBuild object
 at 0xae78a40c>): []
 2015-11-16 20:37:46+0000 [-] BuildStep.failed now firing callback
 2015-11-16 20:37:46+0000 [-]  step 'rpmbuild' complete: exception
 2015-11-16 20:37:46+0000 [-]  <Build rpmbuild>: build finished
 2015-11-16 20:37:46+0000 [-] Starting factory
 <twisted.mail.smtp.ESMTPSenderFactory instance at 0xaea62fec>
 2015-11-16 20:37:46+0000 [-] pruning '/var/lib/broq/master/rpmbuild/1396'
 2015-11-16 20:37:46+0000 [-] pruning '/var/lib/broq/master/rpmbuild/1396
 -log-shell-stdio.bz2'
 2015-11-16 20:37:46+0000 [-] pruning '/var/lib/broq/master/rpmbuild/1396
 -log-rpmbuild-property_changes'
 2015-11-16 20:37:46+0000 [-] pruning '/var/lib/broq/master/rpmbuild/1396
 -log-rpmbuild-stdio.bz2'
 2015-11-16 20:37:46+0000 [-] releaseLocks(<BuildSlave 'karkinos'>): []
 }}}

 Here is a one such exception in more detail:
 {{{
 UnicodeDecodeError('ascii', '\n> jpegtran-bin at 1.0.2 postinstall
 /home/broq/buildbot/karkinos/rpmbuild/build/BUILD/broq-0.5.2.4-1-g2cf5752/build/www/node_modules
 /grunt-contrib-imagemin/node_modules/imagemin/node_modules/imagemin-
 jpegtran/node_modules/jpegtran-bin\n> node lib/install.js\n\n\xe2\x9a\xa0
 pre-build test failed, compiling from source...\n\xe2\x9c\x94 pre-build
 test passed successfully!\n\xe2\x9c\x94 pre-build test passed
 successfully!\n\n> optipng-bin at 1.0.1 postinstall
 /home/broq/buildbot/karkinos/rpmbuild/build/BUILD/broq-0.5.2.4-1-g2cf5752/build/www/node_modules
 /grunt-contrib-imagemin/node_modules/imagemin/node_modules/imagemin-
 optipng/node_modules/optipng-bin\n> node lib/install.js\n\n', 256, 257,
 'ordinal not in range(128)')
 }}}

 Don't know if this is interesting, but the mentioned text appears in the
 middle of the build step run, although exception is thrown when the whole
 build step is finishing.

--
Ticket URL: <http://trac.buildbot.net/ticket/3384>
Buildbot <http://buildbot.net/>
Buildbot: build/test automation