[devel at bb.net] Request for Python 3 porting help: VACUUM and sqlite problem

[Craig Rodrigues]

Hi,

I followed these steps to set up a Python 3.6 build environment:

https://lists.buildbot.net/pipermail/devel/2017-February/012310.html

If ran this test:

trial buildbot.test.unit.test_scripts_cleanupdb.TestCleanupDb.test_cleanup

and got this error:

Traceback (most recent call last):
  File "/Users/crodrigues/buildbot2/master/buildbot/db/pool.py", line 182,
in __thd
    rv = callable(arg, *args, **kwargs)
  File "/Users/crodrigues/buildbot2/master/buildbot/scripts/cleanupdb.py",
line 62, in thd
    r = engine.execute("vacuum;")
  File
"/Users/crodrigues/venv-3.6-2/lib/python3.6/site-packages/sqlalchemy/engine/base.py",
line 939, in execute
    return self._execute_text(object, multiparams, params)
  File
"/Users/crodrigues/venv-3.6-2/lib/python3.6/site-packages/sqlalchemy/engine/base.py",
line 1097, in _execute_text
    statement, parameters
  File
"/Users/crodrigues/venv-3.6-2/lib/python3.6/site-packages/sqlalchemy/engine/base.py",
line 1189, in _execute_context
    context)
  File
"/Users/crodrigues/venv-3.6-2/lib/python3.6/site-packages/sqlalchemy/engine/base.py",
line 1393, in _handle_dbapi_exception
    exc_info

  File
"/Users/crodrigues/venv-3.6-2/lib/python3.6/site-packages/sqlalchemy/util/compat.py",
line 202, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File
"/Users/crodrigues/venv-3.6-2/lib/python3.6/site-packages/sqlalchemy/engine/base.py",
line 1182, in _execute_context
    context)
  File
"/Users/crodrigues/venv-3.6-2/lib/python3.6/site-packages/sqlalchemy/engine/default.py",
line 469, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) cannot VACUUM
from within a transaction [SQL: 'vacuum;']

buildbot.test.unit.test_scripts_cleanupdb.TestCleanupDb.test_cleanup
-------------------------------------------------------------------------------


I did some analysis, and found that the source of the problem is due to this
change in Python 3:
https://docs.python.org/3/whatsnew/3.6.html#changes-in-the-python-api:

* sqlite3 no longer implicitly commits an open transaction before DDL
statements.

I think the easiest way to fix this would be to enable auto-commit mode
for sqlite3, as was the previous default behavior in earlier versions of
Python.

Can someone familiar with the Python sqlite API help me do this?

I don't know where in the buildbot code to do this.

Thanks!

--
Craig
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.buildbot.net/pipermail/devel/attachments/20170131/09a03a17/attachment.html>