Database caching doesn't account for multiple databases

[tiemonster]

Database caching doesn't account for multiple databases. As such, when running unit tests, it attempts to create the cache table twice on the default database connection, causing a unit test failure.

Two database connections were defined: default and datastore. CACHE_BACKEND = "db://cache_table" Error was as follows:

Traceback (most recent call last):
  File "./manage.py", line 11, in <module>
    execute_manager(settings)
  File "/usr/local/lib/python2.6/dist-packages/Django-1.2.1-py2.6.egg/
django/core/management/__init__.py", line 438, in execute_manager
    utility.execute()
  File "/usr/local/lib/python2.6/dist-packages/Django-1.2.1-py2.6.egg/
django/core/management/__init__.py", line 379, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python2.6/dist-packages/Django-1.2.1-py2.6.egg/
django/core/management/base.py", line 191, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/usr/local/lib/python2.6/dist-packages/Django-1.2.1-py2.6.egg/
django/core/management/base.py", line 218, in execute
    output = self.handle(*args, **options)
  File "/usr/local/lib/python2.6/dist-packages/Django-1.2.1-py2.6.egg/
django/core/management/commands/test.py", line 37, in handle
    failures = test_runner.run_tests(test_labels)
  File "/usr/local/lib/python2.6/dist-packages/Django-1.2.1-py2.6.egg/
django/test/simple.py", line 313, in run_tests
    old_config = self.setup_databases()
  File "/usr/local/lib/python2.6/dist-packages/Django-1.2.1-py2.6.egg/
django/test/simple.py", line 270, in setup_databases
    connection.creation.create_test_db(self.verbosity, autoclobber=not
self.interactive)
  File "/usr/local/lib/python2.6/dist-packages/Django-1.2.1-py2.6.egg/
django/db/backends/creation.py", line 358, in create_test_db
    call_command('createcachetable', cache_name)
  File "/usr/local/lib/python2.6/dist-packages/Django-1.2.1-py2.6.egg/
django/core/management/__init__.py", line 166, in call_command
    return klass.execute(*args, **defaults)
  File "/usr/local/lib/python2.6/dist-packages/Django-1.2.1-py2.6.egg/
django/core/management/base.py", line 218, in execute
    output = self.handle(*args, **options)
  File "/usr/local/lib/python2.6/dist-packages/Django-1.2.1-py2.6.egg/
django/core/management/base.py", line 318, in handle
    label_output = self.handle_label(label, **options)
  File "/usr/local/lib/python2.6/dist-packages/Django-1.2.1-py2.6.egg/
django/core/management/commands/createcachetable.py", line 50, in
handle_label
    curs.execute("\n".join(full_statement))
  File "/usr/local/lib/python2.6/dist-packages/Django-1.2.1-py2.6.egg/
django/db/backends/mysql/base.py", line 86, in execute
    return self.cursor.execute(query, args)
  File "/usr/lib/pymodules/python2.6/MySQLdb/cursors.py", line 166, in
execute
    self.errorhandler(self, exc, value)
  File "/usr/lib/pymodules/python2.6/MySQLdb/connections.py", line 35,
in defaulterrorhandler
    raise errorclass, errorvalue
_mysql_exceptions.OperationalError: (1050, "Table 'cache_table'
already exists") 

In my situation, our datastore was a read-only connection. I told Django to treat it as a replication slave using the TEST_MIRROR setting, and the error went away. This won't work for all setups, however. For more information, see discussion on django-users ​here.

[Alex Gaynor]

To me this is a strong argument for moving the DB caching to use the ORM, this is and non-SQL databases. I realize this entails some overhead, but really if you have performance problems you shouldn't be using a DB cache.

[Russell Keith-Magee]

It's not just the overhead I'm concerned about (although that is a mild concern). It's the mechanics of getting a model installed.

• The database table name is only available via settings.py
• and only then *if* the database cache backend is being used;
• there isn't a models.py to put the model file in.

I suppose the other approach would be to use the router, but use a dummy model that can be clearly identified - i.e, a class that quacks like a normal Django model, but is actually a representation of the database cache model. The 'cache model' wouldn't actually be used for database operations; it would just provide the metadata that the router requires. That way we step outside the usual app/models registration process, but can use the simple interface for routing purposes.

[Russell Keith-Magee]

(In [13473]) Fixed #13946 -- Modified the database cache backend to use the database router to determine availability of the cache table. Thanks to tiemonster for the report.

[Russell Keith-Magee]

(In [13474]) [1.2.X] Fixed #13946 -- Modified the database cache backend to use the database router to determine availability of the cache table. Thanks to tiemonster for the report.

Backport of r13473 from trunk.

[Jacob]

Milestone 1.3 deleted