Unable to create model instance after changing an abstract model to non-abstract due to "foreign key mismatch" error

[josephbiko]

Django Foreign key's do not track the models DB when the model is inherited from abstract class.

Steps to reproduce:

create models:"

class A(models.Model):
    field = models.IntegerField()
    
    class Meta:
        abstract = True

class B(A):
    field2 = models.IntegerField()

class C(models.Model):
    fk = models.ForeignKey(B,on_delete=models.CASCADE)

migrate

change the models (by removeing the meta )to:

class A(models.Model):
    field = models.IntegerField()


class B(A):
    field2 = models.IntegerField()

class C(models.Model):
    fk = models.ForeignKey(B,on_delete=models.CASCADE)

run migrations.

Now add an object of class B in the admin.

resulting error:
"foreign key mismatch - "models_c" referencing "models_b""

[Simon Charette]

Hello josephbiko,

I assume you meant that running makemigrations between your two model scenarios doesn't generate the appropriate database alterations?

There's a few bugs (#23521, #25247) related to switching from concrete to abstract inheritance but in the abstract to concrete case there shouldn't be any problem as C.fkshould be automatically repointed to B.a_ptr when A is made concrete.

I'm afraid we'll need a more detailed reproduction test case in order to reproduce your issue. Please include tracebacks and detail every steps your perform to get there.

[Simon Charette]

Upon further inspection I assume you had an existing B entry before attempting to run the migration that adds the B.a_ptr primary key and it failed because no rows were present in the newly added A table?

In this case I think you should be in charge of populating A with a data migration (either using RunPython or RunSQL) instead of expecting Django to do it for you.

[josephbiko]

Replying to Simon Charette:

Upon further inspection I assume you had an existing B entry before attempting to run the migration that adds the B.a_ptr primary key and it failed because no rows were present in the newly added A table?

In this case I think you should be in charge of populating A with a data migration (either using RunPython or RunSQL) instead of expecting Django to do it for you.

No there was no data present before

[Tim Graham]

Reproduced at 1e9b02a4c28142303fb4d33632e77ff7e26acb8b.

[Bhuvnesh]

hi, i was working around to find the real cause of this issue and i am writing my observation here but i am not sure if they are fully correct or not, it would be really helpful if someone could cross-check/confirm.
1. when a_ptr is created its a primary key hence has UNIQUE constraint.
2. since a_ptr is a non nullable field it asks for a default value to populate the existing rows(even for empty db). And as we provide default value the UNIQUE constraint fails.
3. Now as UNIQUE constraints failed , it defies one of the rule of foreign key constraints given ​here (Under the heading Required and Suggested Database Indexes) which says The parent key columns named in the foreign key constraint are not the primary key of the parent table and are not subject to a unique constraint using collating sequence specified in the CREATE TABLE. This subsequently leads to the "foreign key mismatch - "models_c" referencing "models_b"" error.

I'm using django dev version and its giving me error as soon as i migrate second time.

[Bhuvnesh]

Ok, so i have found the cause (the one above is incorrect):

1. When we run migration for the first time 2 db tables are created (B and C) , in which C.fk points to primary key of B i.e id .
2. Now when we run second migration, table A is created and B is recreated with field a_ptr as primary key and all the old fields of B are deleted(evenid). No changes for table C.
3. Since c.fk still points to b.id (at physical level), it throws foreign_key_mismatch error.
4. If we explicity define primary key in model B everything works as expected.(temporary solution)

   bid = models.IntegerField(primary_key=True)
   

Maybe a rough solution can be to recreate a model if new primary key is created and model contains a related field?

[Bhuvnesh]

WIP PR ​https://github.com/django/django/pull/16325

[Calvin Vu]

I have reproduced the foreign key mismatch error but with slightly different steps:

1. Add the models to models.py

   class A(models.Model):
       field = models.IntegerField()
       
       class Meta:
           abstract = True
   
   class B(A):
       field2 = models.IntegerField()
   
   class C(models.Model):
       fk = models.ForeignKey(B,on_delete=models.CASCADE)
   

2. Make migrations and migrate
3. Make the A model concrete (by removing its Meta)

   class A(models.Model):
       field = models.IntegerField()
   
   class B(A):
       field2 = models.IntegerField()
   
   class C(models.Model):
       fk = models.ForeignKey(B, on_delete=models.CASCADE)
   

4. Make migrations which will present the following output:

   It is impossible to add a non-nullable field 'a_ptr' to b without specifying a default. This is because the database needs something to populate existing rows.
   Please select a fix:
    1) Provide a one-off default now (will be set on all existing rows with a null value for this column)
    2) Quit and manually define a default value in models.py.
   Select an option: 
   

5. Go with option 1 and provide an integer (I provided 0)
6. Migrate and you'll see the foreign key mismatch error

[Calvin Vu]

It seems like Bhuvnesh correctly identified the cause. A potential solution would be to first set C.fk to A.id (which is going to be the id of model 'AB', i.e., the separate tables A and B, which are connected one-to-one with B.a_ptr_id when model A is made concrete), then change model B to inherit from model A (to form model 'AB'), and then update any existing data. Alternatively, we could also set C.fk to B.a_ptr_id as it is also a primary key.

[Calvin Vu]

Replying to Calvin Vu:

It seems like Bhuvnesh correctly identified the cause. A potential solution would be to first set C.fk to A.id (which is going to be the id of model 'AB', i.e., the separate tables A and B, which are connected one-to-one with B.a_ptr_id when model A is made concrete), then change model B to inherit from model A (to form model 'AB'), and then update any existing data. Alternatively, we could also set C.fk to B.a_ptr_id as it is also a primary key.

Nevermind, this solution probably won't work because we need B to inherit from A to determine exactly which A instance to set C.fk to but when B inherits from A we lose B.id so there would be no way to link C and 'AB'