Adding database dynamically breaks in Django 4.1+

[Dhruva Patil]

Adding databases to django.conf.settings.DATABASES was working properly in django < 4.1. Since 4.1, it has started breaking.

Traceback (most recent call last):
  File "/home/appuser/.local/lib/python3.8/site-packages/django/core/handlers/exception.py", line 55, in inner
    response = get_response(request)
  File "/home/appuser/.local/lib/python3.8/site-packages/django/core/handlers/base.py", line 197, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/home/appuser/.local/lib/python3.8/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
    return view_func(*args, **kwargs)
  File "/home/appuser/.local/lib/python3.8/site-packages/rest_framework/viewsets.py", line 125, in view
    return self.dispatch(request, *args, **kwargs)
  File "/home/appuser/.local/lib/python3.8/site-packages/rest_framework/views.py", line 509, in dispatch
    response = self.handle_exception(exc)
  File "/home/appuser/.local/lib/python3.8/site-packages/rest_framework/views.py", line 469, in handle_exception
    self.raise_uncaught_exception(exc)
  File "/home/appuser/.local/lib/python3.8/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception
    raise exc
  File "/home/appuser/.local/lib/python3.8/site-packages/rest_framework/views.py", line 506, in dispatch
    response = handler(request, *args, **kwargs)
  ...
  File "/home/appuser/.local/lib/python3.8/site-packages/django/db/models/query.py", line 605, in aggregate
    return query.get_aggregation(self.db, kwargs)
  File "/home/appuser/.local/lib/python3.8/site-packages/django/db/models/sql/query.py", line 544, in get_aggregation
    result = compiler.execute_sql(SINGLE)
  File "/home/appuser/.local/lib/python3.8/site-packages/django/db/models/sql/compiler.py", line 1393, in execute_sql
    cursor = self.connection.cursor()
  File "/home/appuser/.local/lib/python3.8/site-packages/django/utils/asyncio.py", line 26, in inner
    return func(*args, **kwargs)
  File "/home/appuser/.local/lib/python3.8/site-packages/django/db/backends/base/base.py", line 323, in cursor
    return self._cursor()
  File "/home/appuser/.local/lib/python3.8/site-packages/django/db/backends/base/base.py", line 299, in _cursor
    self.ensure_connection()
  File "/home/appuser/.local/lib/python3.8/site-packages/django/utils/asyncio.py", line 26, in inner
    return func(*args, **kwargs)
  File "/home/appuser/.local/lib/python3.8/site-packages/django/db/backends/base/base.py", line 282, in ensure_connection
    self.connect()
  File "/home/appuser/.local/lib/python3.8/site-packages/django/utils/asyncio.py", line 26, in inner
    return func(*args, **kwargs)
  File "/home/appuser/.local/lib/python3.8/site-packages/django/db/backends/base/base.py", line 247, in connect
    self.check_settings()
  File "/home/appuser/.local/lib/python3.8/site-packages/django/db/backends/base/base.py", line 271, in check_settings
    if self.settings_dict["TIME_ZONE"] is not None and not settings.USE_TZ:
KeyError: 'TIME_ZONE'

[Mariusz Felisiak]

Creating a new database connection or modifying an existing one on fly has never been supported.

[Dhruva Patil]

Replying to Mariusz Felisiak:

Creating a new database connection or modifying an existing one on fly has never been supported.

We've been using Django for our multi tenant app to support new db connections on the fly, along with a middleware and it has worked quite well, with few hiccups due to the "unofficial" nature of the implementation. Django 4.1 broke this for seemingly no good reason.

If the philosophy is to reject making even minor changes to the app to retain functionality that was unofficially available, then that is your prerogative, you can close the ticket.

However, it would be ideal to retain the behaviour of the app for such scenarios that was available in Django before 4.1. As the code changes required for it is very minimal, I would prefer this option. I have raised a PR for the same: ​https://github.com/django/django/pull/16647

[Mariusz Felisiak]

We've been using Django for our multi tenant app to support new db connections on the fly, along with a middleware and it has worked quite well, with few hiccups due to the "unofficial" nature of the implementation. Django 4.1 broke this for seemingly no good reason.

If the philosophy is to reject making even minor changes to the app to retain functionality that was unofficially available, then that is your prerogative, you can close the ticket.

We rejected such tickets in the past, see #33578.

However, it would be ideal to retain the behaviour of the app for such scenarios that was available in Django before 4.1. As the code changes required for it is very minimal, I would prefer this option. I have raised a PR for the same: ​https://github.com/django/django/pull/16647

Unfortunately, the number of lines changed does not matter, as we cannot guarantee that we will not break it again in the future.

[Carlton Gibson]

  File "/home/appuser/.local/lib/python3.8/site-packages/django/db/backends/base/base.py", line 271, in check_settings
    if self.settings_dict["TIME_ZONE"] is not None and not settings.USE_TZ:
KeyError: 'TIME_ZONE'

Did you try making sure {"TIME_ZONE": None} appears in your settings dict?

[Dhruva Patil]

Replying to Carlton Gibson:

  File "/home/appuser/.local/lib/python3.8/site-packages/django/db/backends/base/base.py", line 271, in check_settings
    if self.settings_dict["TIME_ZONE"] is not None and not settings.USE_TZ:
KeyError: 'TIME_ZONE'

Did you try making sure {"TIME_ZONE": None} appears in your settings dict?

Need to add all the default parameters, that works. However, we will now have to maintain that separately. It's not a major change as such, we were just hoping to avoid having ancillary code that was already part of the source code before for this to work.

[Florian Apolloner]

On principle I am not opposed to fixing this (I know I am using Django for things much worse :D). That said only as long as it is clear that we don't see it as supported configuration.