When time zone support is enabled, under Windows, cache keys may contain non-ASCII characters

[Aymeric Augustin]

Reported by Ramiro:

d:\proj\django\trunk\tests>set PYTHONPATH=d:\proj\django\trunk
 
d:\proj\django\trunk\tests>c:\python25\python runtests.py --settings=test_sqlite cache
Creating test database for alias 'default'...
Creating test database for alias 'other'...
..............................................E.................................
...E............................................................................
............................................ssssssssssssssssssssssssssssssssss
======================================================================
ERROR: test_cache_write_unpickable_object (regressiontests.cache.tests.DBCacheTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "d:\proj\django\trunk\tests\regressiontests\cache\tests.py", line 773, in test_cache_write_unpickable_object
    get_cache_data = fetch_middleware.process_request(request)
  File "d:\proj\django\trunk\django\middleware\cache.py", line 143, in process_request
    cache_key = get_cache_key(request, self.key_prefix, 'GET', cache=self.cache)
  File "d:\proj\django\trunk\django\utils\cache.py", line 207, in get_cache_key
    headerlist = cache.get(cache_key, None)
  File "d:\proj\django\trunk\django\core\cache\backends\db.py", line 62, in get
    "WHERE cache_key = %%s" % table, [key])
  File "d:\proj\django\trunk\django\db\backends\sqlite3\base.py", line 321, in execute
    return Database.Cursor.execute(self, query, params)
  File "d:\proj\django\trunk\django\db\backends\sqlite3\base.py", line 38, in parse_datetime_with_timezone_support
    dt = parse_datetime(value)
  File "d:\proj\django\trunk\django\utils\dateparse.py", line 70, in parse_datetime
    match = datetime_re.match(value)
DatabaseError: Could not decode to UTF-8 column 'cache_key' with text
  ':1:views.decorators.cache.cache_header..0b2d3961361a3afb82d85e3b25762eab.en.Hora_estßndar_de_Argentina'
 
======================================================================
ERROR: test_cache_write_unpickable_object (regressiontests.cache.tests.DBCacheTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "d:\proj\django\trunk\tests\regressiontests\cache\tests.py", line 773, in test_cache_write_unpickable_object
    get_cache_data = fetch_middleware.process_request(request)
  File "d:\proj\django\trunk\django\middleware\cache.py", line 143, in process_request
    cache_key = get_cache_key(request, self.key_prefix, 'GET', cache=self.cache)
  File "d:\proj\django\trunk\django\utils\cache.py", line 207, in get_cache_key
    headerlist = cache.get(cache_key, None)
  File "d:\proj\django\trunk\django\core\cache\backends\db.py", line 62, in get
    "WHERE cache_key = %%s" % table, [key])
  File "d:\proj\django\trunk\django\db\backends\sqlite3\base.py", line 321, in execute
    return Database.Cursor.execute(self, query, params)
  File "d:\proj\django\trunk\django\db\backends\sqlite3\base.py", line 38, in parse_datetime_with_timezone_support
    dt = parse_datetime(value)
  File "d:\proj\django\trunk\django\utils\dateparse.py", line 70, in parse_datetime
    match = datetime_re.match(value)
DatabaseError: Could not decode to UTF-8 column 'cache_key' with text
  ':1:views.decorators.cache.cache_header..0b2d3961361a3afb82d85e3b25762eab.en.Hora_estßndar_de_Argentina'
 
----------------------------------------------------------------------
Ran 238 tests in 32.209s
 
FAILED (errors=2, skipped=34)
Destroying test database for alias 'default'...
Destroying test database for alias 'other'...
 
 
d:\proj\django\trunk\tests>c:\python27\python runtests.py --settings=test_sqlite
 cache
Creating test database for alias 'default'...
Creating test database for alias 'other'...
..............................................E.................................
...E............................................................................
............................................ssssssssssssssssssssssssssssssssss
======================================================================
ERROR: test_cache_write_unpickable_object (regressiontests.cache.tests.DBCacheTe
sts)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "d:\proj\django\trunk\tests\regressiontests\cache\tests.py", line 771, in test_cache_write_unpickable_object
    update_middleware.process_response(request, response)
  File "d:\proj\django\trunk\django\middleware\cache.py", line 109, in process_response
    cache_key = learn_cache_key(request, response, timeout, self.key_prefix, cache=self.cache)
  File "d:\proj\django\trunk\django\utils\cache.py", line 241, in learn_cache_key
    cache.set(cache_key, [], cache_timeout)
  File "d:\proj\django\trunk\django\core\cache\backends\db.py", line 80, in set
    self._base_set('set', key, value, timeout)
  File "d:\proj\django\trunk\django\core\cache\backends\db.py", line 108, in _base_set
    "WHERE cache_key = %%s" % table, [key])
  File "d:\proj\django\trunk\django\db\backends\sqlite3\base.py", line 321, in execute
    return Database.Cursor.execute(self, query, params)
DatabaseError: You must not use 8-bit bytestrings unless you use a text_factory
  that can interpret 8-bit bytestrings (like text_factory = str). It is highly
  recommended that you instead just switch your application to Unicode strings.
 
======================================================================
ERROR: test_cache_write_unpickable_object (regressiontests.cache.tests.DBCacheTe
sts)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "d:\proj\django\trunk\tests\regressiontests\cache\tests.py", line 771, in test_cache_write_unpickable_object
    update_middleware.process_response(request, response)
  File "d:\proj\django\trunk\django\middleware\cache.py", line 109, in process_response
    cache_key = learn_cache_key(request, response, timeout, self.key_prefix, cache=self.cache)
  File "d:\proj\django\trunk\django\utils\cache.py", line 241, in learn_cache_key
    cache.set(cache_key, [], cache_timeout)
  File "d:\proj\django\trunk\django\core\cache\backends\db.py", line 80, in set
    self._base_set('set', key, value, timeout)
  File "d:\proj\django\trunk\django\core\cache\backends\db.py", line 108, in _base_set
    "WHERE cache_key = %%s" % table, [key])
  File "d:\proj\django\trunk\django\db\backends\sqlite3\base.py", line 321, in execute
    return Database.Cursor.execute(self, query, params)
DatabaseError: You must not use 8-bit bytestrings unless you use a text_factory
  that can interpret 8-bit bytestrings (like text_factory = str). It is highly
  recommended that you instead just switch your application to Unicode strings.
 
----------------------------------------------------------------------
Ran 238 tests in 26.808s
 
FAILED (errors=2, skipped=34)
Destroying test database for alias 'default'...
Destroying test database for alias 'other'...

[Aymeric Augustin]

In [17286]:

Fixed #17476 -- Ensure timezone-dependant cache keys only use ASCII characters, especially on Windows.

[Tim Graham]

I think this fix may be obsolete as I tried adding @override_settings(TIME_ZONE='Hora_estándar_de_Argentina') to test_cache_write_unpicklable_object (one of the failing tests in this report). Even on Windows, that gave an UnknownTimeZoneError from pytz. ​PR

[Tim Graham <timograham@…>]

In 4ff29a5:

Refs #17476 -- Removed obsolete simplification of timezone names in cache key generation.