[Buildbot-devel] 8-bit chars vs. sqlalchemy?

[Jeff Rizzo]

In troubleshooting why a particular commit didn't trigger a build, I 
came across this in twistd.log:

sqlalchemy.exc.ProgrammingError: (ProgrammingError) You must not use 
8-bit bytestrings unless you use a text_factory that can interpret 8-bit 
bytestrings (like text_factory = str). It is highly recommended that you 
instead just switch your application to Unicode strings. u'INSERT INTO 
changes (author, comments, is_dir, branch, revision, revlink, 
when_timestamp, category, repository, codebase, project) VALUES (?, ?, 
?, ?, ?, ?, ?, ?, ?, ?, ?)' ('bouyer', "XS_NOERROR | XS_RESET doens't 
make sense, XS_RESET is what is wanted here.\nPointed out by Edgar 
Fu\xdf\n", 0, None, '2013-11-24 18:02:08', '', 1385316128, 'kernel.', 
'anoncvs at anoncvs.netbsd.org:/cvsroot', '', 'NetBSD')


(full traceback is below)

Clearly, the commit log with the \xdf character is at issue - is there 
anything simple I can do in my changesource to avoid this?  I would 
prefer (obviously) not to strip out non-ascii characters, but I'm not 
the most unicode-savvy...


Full traceback:

2013-11-24 18:02:28+0000 [-] while reading 
'1385316139.12111_0.build.tastylime.net' from maildir '/home/buildbot/Mail':
         Traceback (most recent call last):
           File 
"/usr/pkg/lib/python2.7/site-packages/twisted/internet/defer.py", line 
575, in _runCallbacks
             current.result = callback(current.result, *args, **kw)
           File 
"/usr/pkg/lib/python2.7/site-packages/twisted/internet/defer.py", line 
1126, in gotResult
             _inlineCallbacks(r, g, deferred)
           File 
"/usr/pkg/lib/python2.7/site-packages/twisted/internet/defer.py", line 
1068, in _inlineCallbacks
             result = result.throwExceptionIntoGenerator(g)
           File 
"/usr/pkg/lib/python2.7/site-packages/twisted/python/failure.py", line 
389, in throwExceptionIntoGenerator
             return g.throw(self.type, self.value, self.tb)
         --- <exception caught here> ---
           File 
"/usr/pkg/lib/python2.7/site-packages/buildbot/util/maildir.py", line 
116, in poll
             yield self.messageReceived(n)
           File 
"/usr/pkg/lib/python2.7/site-packages/twisted/python/threadpool.py", 
line 191, in _worker
             result = context.call(ctx, function, *args, **kwargs)
           File 
"/usr/pkg/lib/python2.7/site-packages/twisted/python/context.py", line 
118, in callWithContext
             return self.currentContext().callWithContext(ctx, func, 
*args, **kw)
           File 
"/usr/pkg/lib/python2.7/site-packages/twisted/python/context.py", line 
81, in callWithContext
             return func(*args,**kw)
           File 
"/usr/pkg/lib/python2.7/site-packages/buildbot/db/pool.py", line 184, in 
__thd
             rv = callable(arg, *args, **kwargs)
           File 
"/usr/pkg/lib/python2.7/site-packages/buildbot/db/changes.py", line 78, 
in thd
             project=project))
           File 
"/usr/pkg/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 
1449, in execute
             params)
           File 
"/usr/pkg/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 
1584, in _execute_clauseelement
             compiled_sql, distilled_params
           File 
"/usr/pkg/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 
1698, in _execute_context
             context)
           File 
"/usr/pkg/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 
1691, in _execute_context
             context)
           File 
"/usr/pkg/lib/python2.7/site-packages/sqlalchemy/engine/default.py", 
line 331, in do_execute
             cursor.execute(statement, parameters)
         sqlalchemy.exc.ProgrammingError: (ProgrammingError) You must 
not use 8-bit bytestrings unless you use a text_factory that can 
interpret 8-bit bytestrings (like text_factory = str). It is highly 
recommended that you instead just switch your application to Unicode 
strings. u'INSERT INTO changes (author, comments, is_dir, branch, 
revision, revlink, when_timestamp, category, repository, codebase, 
project) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)' ('bouyer', 
"XS_NOERROR | XS_RESET doens't make sense, XS_RESET is what is wanted 
here.\nPointed out by Edgar Fu\xdf\n", 0, None, '2013-11-24 18:02:08', 
'', 1385316128, 'kernel.', 'anoncvs at anoncvs.netbsd.org:/cvsroot', '', 
'NetBSD')