#33280 closed Bug (invalid)
Migration with django-database-view don't work with multiple databases. — at Version 3
Reported by: | razielvamp666 | Owned by: | nobody |
---|---|---|---|
Component: | Database layer (models, ORM) | Version: | 2.2 |
Severity: | Normal | Keywords: | orm migration |
Cc: | Triage Stage: | Unreviewed | |
Has patch: | no | Needs documentation: | no |
Needs tests: | no | Patch needs improvement: | no |
Easy pickings: | no | UI/UX: | no |
Description (last modified by )
My configuration is:
python 3.7.1
Django==2.2.24 django-axes==5.16.0 django-bootstrap-modal-forms==1.3.1 django-celery-results==1.0.4 django-cors-headers==3.1.1 django-database-view==0.3.0 django-debug-toolbar==1.11 django-dynamic-db-router==0.3.0 django-ipware==3.0.7 django-redis-cache==3.0.0 django-redis-sessions==0.6.2 django-ses==1.0.3 django-storages==1.12.3 django-taggit==0.24.0 djangorestframework==3.12.4 pytest-django==4.4.0
I using Django with multiple databases and default DB is not set for security reasons (I prefer to get error if Django will fallback to default
connection, because it means that current tenant DB is not used for some reasons). so default
connection is empty ({}
).
When I run python manage.py migrate --database=some_db_name
I've get error:
Applying app_name.0154_migration_name...Traceback (most recent call last): File "manage.py", line 41, in <module> execute_from_command_line(sys.argv) File "/user/.pyenv/versions/gl/lib/python3.7/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line utility.execute() File "/user/.pyenv/versions/gl/lib/python3.7/site-packages/django/core/management/__init__.py", line 375, in execute self.fetch_command(subcommand).run_from_argv(self.argv) File "/user/.pyenv/versions/gl/lib/python3.7/site-packages/django/core/management/base.py", line 323, in run_from_argv self.execute(*args, **cmd_options) File "/user/.pyenv/versions/gl/lib/python3.7/site-packages/django/core/management/base.py", line 364, in execute output = self.handle(*args, **options) File "/user/.pyenv/versions/gl/lib/python3.7/site-packages/django/core/management/base.py", line 83, in wrapped res = handle_func(*args, **kwargs) File "/user/.pyenv/versions/gl/lib/python3.7/site-packages/django/core/management/commands/migrate.py", line 234, in handle fake_initial=fake_initial, File "/user/.pyenv/versions/gl/lib/python3.7/site-packages/django/db/migrations/executor.py", line 117, in migrate state = self._migrate_all_forwards(state, plan, full_plan, fake=fake, fake_initial=fake_initial) File "/user/.pyenv/versions/gl/lib/python3.7/site-packages/django/db/migrations/executor.py", line 147, in _migrate_all_forwards state = self.apply_migration(state, migration, fake=fake, fake_initial=fake_initial) File "/user/.pyenv/versions/gl/lib/python3.7/site-packages/django/db/migrations/executor.py", line 245, in apply_migration state = migration.apply(state, schema_editor) File "/user/.pyenv/versions/gl/lib/python3.7/site-packages/django/db/migrations/migration.py", line 124, in apply operation.database_forwards(self.app_label, schema_editor, old_state, project_state) File "/user/.pyenv/versions/gl/lib/python3.7/site-packages/dbview/helpers.py", line 21, in database_forwards self._create_standard_view(model, schema_editor) File "/user/.pyenv/versions/gl/lib/python3.7/site-packages/dbview/helpers.py", line 60, in _create_standard_view qs = str(model.view()) File "/user/code/app_name/app_name/models/file_name.py", line 136, in view return str(qs.query) File "/user/.pyenv/versions/gl/lib/python3.7/site-packages/django/db/models/sql/query.py", line 265, in __str__ sql, params = self.sql_with_params() File "/user/.pyenv/versions/gl/lib/python3.7/site-packages/django/db/models/sql/query.py", line 273, in sql_with_params return self.get_compiler(DEFAULT_DB_ALIAS).as_sql() File "/user/.pyenv/versions/gl/lib/python3.7/site-packages/django/db/models/sql/compiler.py", line 474, in as_sql extra_select, order_by, group_by = self.pre_sql_setup() File "/user/.pyenv/versions/gl/lib/python3.7/site-packages/django/db/models/sql/compiler.py", line 54, in pre_sql_setup self.setup_query() File "/user/.pyenv/versions/gl/lib/python3.7/site-packages/django/db/models/sql/compiler.py", line 45, in setup_query self.select, self.klass_info, self.annotation_col_map = self.get_select() File "/user/.pyenv/versions/gl/lib/python3.7/site-packages/django/db/models/sql/compiler.py", line 254, in get_select sql, params = self.compile(col, select_format=True) File "/user/.pyenv/versions/gl/lib/python3.7/site-packages/django/db/models/sql/compiler.py", line 405, in compile sql, params = node.as_sql(self, self.connection) File "/user/.pyenv/versions/gl/lib/python3.7/site-packages/django/db/models/expressions.py", line 737, in as_sql return "%s.%s" % (qn(self.alias), qn(self.target.column)), [] File "/user/.pyenv/versions/gl/lib/python3.7/site-packages/django/db/models/sql/compiler.py", line 396, in quote_name_unless_alias r = self.connection.ops.quote_name(name) File "/user/.pyenv/versions/gl/lib/python3.7/site-packages/django/db/backends/dummy/base.py", line 20, in complain raise ImproperlyConfigured("settings.DATABASES is improperly configured. " django.core.exceptions.ImproperlyConfigured: settings.DATABASES is improperly configured. Please supply the ENGINE value. Check settings documentation for more details.
You can see that problem is here:
return self.get_compiler(DEFAULT_DB_ALIAS).as_sql()
DEFAULT_DB_ALIAS
is default
and it hardcoded.
Also DEFAULT_DB_ALIAS
is used couple of times inside django/db/models/sql/query.py
(please search by DEFAULT_DB_ALIAS
in file to see details)
It should use CURRENT_DB as other parts of Django
Unfortunately I didn't check how migration determines current DB when --database=...
is set so can't recommend exact solution, but I may see couple hints:
- DB may be get from
--database=...
directly (But not sure if that is a correct solution). - DB may be get from [
DatabaseRouter
](https://docs.djangoproject.com/en/2.2/topics/db/multi-db/#database-routers) as other parts of Django (It is looking like correct solution) - Here should be
using
option to be able set connection DB directly (for example, all functions insidedjango/db/transaction.py
hasusing
argument to be able set connection name directly)
Right now I using workaround and set from django.conf import settings; settings.DATABASES['default'] = current_db_taken_from_cmd
if command is migrate
.
Not all migrations fail. I don't know reason why it happens only with specific migrations, but I guess that it because this migration doing CreateView
Migration file is looking like:
# Generated by Django 2.2.10 on 2021-10-06 04:17 from dbview.helpers import CreateView import common.enums.some_enum from django.db import migrations, models import django.db.models.deletion class Migration(migrations.Migration): dependencies = [ ('app_name', '0153_previous_migration'), ] operations = [ migrations.CreateModel( name='SomeName', fields=[ ('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')), ('create_datetime', models.DateTimeField(auto_now_add=True, verbose_name='A')), ('user', models.CharField(default='admin', max_length=32, verbose_name='B')), ('update_datetime', models.DateTimeField(auto_now=True, verbose_name='C')), ... ], options={ 'db_table': 'table_name', 'unique_together': {('other_table', 'table_name')}, }, ), CreateView( name='SomeNameView', fields=[ ('table_name_1', models.OneToOneField(on_delete=django.db.models.deletion.PROTECT, primary_key=True, serialize=False, to='orm_django.ApplyingInformation')), ('table_name_2', models.IntegerField(verbose_name='A')), ('table_name_3', models.IntegerField(verbose_name='B')), ('table_name_3', models.IntegerField(verbose_name='C')), ... ], options={ 'db_table': 'table_view', }, ), migrations.AddIndex( model_name='modelname', index=models.Index(fields=['field_A', 'field_B', ...], name='index_name'), ), ]
Change History (3)
comment:1 by , 3 years ago
Resolution: | → invalid |
---|---|
Status: | new → closed |
Summary: | [bug] migration don't work with multiple databases (sometimes) → Migration with django-database-view don't work with multiple databases. |
comment:2 by , 3 years ago
Description: | modified (diff) |
---|
comment:3 by , 3 years ago
Description: | modified (diff) |
---|
Thanks for this report. However a lot of things happen outside of Django, you have many 3rd-party packages, a custom
Operation
in migrations, etc. The main issue is that you're using thequery
attribute to generate an SQL which is not recommended, documented, or supported. Moreover, this helper has issues with parameters (see #25705) and doesn't work when thedefault
database is empty (see #25947). I'm not marking this as a duplicate of #25947 as fixing it will not make.query
the recommended way to generate an SQL. I would use a raw SQL in your case.