Opened 3 years ago

Closed 3 years ago

Last modified 3 years ago

#33462 closed Bug (fixed)

Altering type of referenced primary key with MTI causes duplicated operations.

Reported by: bcail Owned by: Mariusz Felisiak
Component: Migrations Version: 4.0
Severity: Release blocker Keywords:
Cc: David Wobrock Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

I ran into an intermittent migration issue on Django 4.x and PostgreSQL.
The migrations work fine on Django 3.2.x, but intermittently fail on
Django 4.x with the following exception:

django.db.utils.ProgrammingError: constraint
"polls_questioncontribution_base_question_id_25bfb2a8_fk" for relation
"polls_questioncontribution" already exists

I made a small demo repo to show the problem:
https://github.com/bcail/migration_error. I put the sqlmigrate output
from Django 3.2 and Django 4 in the repo, and it looks like Django 4
(sometimes) creates duplicate SQL statements.

I also ran git bisect on the Django code, and it looks like the problem
starts showing up with this commit:
https://github.com/django/django/commit/3d9040a50b160f8b4bb580e09f4120d4979fe29e.

Change History (7)

comment:1 by Pedro Schlickmann Mendes, 3 years ago

Triage Stage: UnreviewedAccepted

Succesfully reproduced it. But I'm still trying to figure it out why this happens only sometimes for you. For me it happens everytime I try to run the 0008 migration for the first time, the second time it runs perfectly. The same happens when I try to revert it to the 0004 migration, the first time fails, and the second time it runs perfectly

comment:2 by bcail, 3 years ago

Using "python manage.py test -v 2" to run the migrations on a test db, it fails for me pretty often. However, it did succeed after I restarted postgresql (and then it failed again). Another time after I restarted postgresql, it failed the first time.

comment:3 by Mariusz Felisiak, 3 years ago

Cc: David Wobrock added
Severity: NormalRelease blocker
Summary: intermittent migration errorAltering type of referenced primary key with MTI causes duplicated operations.
Version: dev4.0

Thanks for the report! This is caused by multiple references to the same table (MTI and foreign keys). I was able to fix this by excluding relations that were already yielded:

  • django/db/backends/base/schema.py

    diff --git a/django/db/backends/base/schema.py b/django/db/backends/base/schema.py
    index 47fdae8fd4..c057092564 100644
    a b def _all_related_fields(model):  
    3333    return model._meta._get_fields(forward=False, reverse=True, include_hidden=True)
    3434
    3535
    36 def _related_non_m2m_objects(old_field, new_field):
     36def _related_non_m2m_objects(old_field, new_field, excluded=None):
    3737    # Filter out m2m objects from reverse relations.
    3838    # Return (old_relation, new_relation) tuples.
    3939    related_fields = zip(
    4040        (obj for obj in _all_related_fields(old_field.model) if _is_relevant_relation(obj, old_field)),
    4141        (obj for obj in _all_related_fields(new_field.model) if _is_relevant_relation(obj, new_field)),
    4242    )
     43    excluded = excluded or ()
    4344    for old_rel, new_rel in related_fields:
    44         yield old_rel, new_rel
    45         yield from _related_non_m2m_objects(
    46             old_rel.remote_field,
    47             new_rel.remote_field,
    48         )
     45        if (old_rel, new_rel) not in excluded:
     46            yield old_rel, new_rel
     47            yield from _related_non_m2m_objects(
     48                old_rel.remote_field,
     49                new_rel.remote_field,
     50                excluded=(*related_fields, *excluded),
     51            )
    4952
    5053
    5154class BaseDatabaseSchemaEditor:

I'm not 100% satisfied with this patch.

This is really a regression in 325d7710ce9f6155bb55610ad6b4580d31263557.

comment:4 by Mariusz Felisiak, 3 years ago

Owner: changed from nobody to Mariusz Felisiak
Status: newassigned

comment:5 by Mariusz Felisiak, 3 years ago

Has patch: set

I found an easier solution, PR.

comment:6 by GitHub <noreply@…>, 3 years ago

Resolution: fixed
Status: assignedclosed

In e972620a:

Fixed #33462 -- Fixed migration crash when altering type of primary key with MTI and foreign key.

This prevents duplicated operations when altering type of primary key
with MTI and foreign key. Previously, a foreign key to the base model
was added twice, once directly and once by the inheritance model.

Thanks bcail for the report.

Regression in 325d7710ce9f6155bb55610ad6b4580d31263557.

comment:7 by Mariusz Felisiak <felisiak.mariusz@…>, 3 years ago

In 7c2d4d94:

[4.0.x] Fixed #33462 -- Fixed migration crash when altering type of primary key with MTI and foreign key.

This prevents duplicated operations when altering type of primary key
with MTI and foreign key. Previously, a foreign key to the base model
was added twice, once directly and once by the inheritance model.

Thanks bcail for the report.

Regression in 325d7710ce9f6155bb55610ad6b4580d31263557.
Backport of e972620ada4f9ed7bc57f28e133e85c85b0a7b20 from main

Note: See TracTickets for help on using tickets.
Back to Top