Running migrations when updating default_auto_field only updates primary key.

[greemo]

I just added the AppConfig.default_auto_field setting to one of my apps and created migrations. I have many to one relationships, and it seems when I ran the makemigrations, it only updated the primary key side of the migration.

I am running on a MySQL server, and it is not allowed to have foreign keys with different INT size to primary keys, so the migration can not be applied. This mismatch is allowed in postgres, but not recommended.

This is going to require some painful hand-creation of migrations where I drop all foreign key constraints on foreign keys in one migration, then recreate in another.

This seems like it could be done automatically by the django migration system when it sees the int size of an auto-generated primary key is increased, to then increase the size of the foreign key to match.

I have also tried changing it is settings via the following, which produced the same results:

DEFAULT_AUTO_FIELD = 'django.db.models.BigAutoField'

The code running this is juntagrico: ​https://github.com/juntagrico/juntagrico

The migration produced is as follows:

# Generated by Django 3.2 on 2021-05-12 08:00

from django.db import migrations, models


class Migration(migrations.Migration):

    dependencies = [
        ('juntagrico', '0033_post_1_4'),
    ]

    operations = [
        migrations.AlterField(
            model_name='activityarea',
            name='id',
            field=models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID'),
        ),
        migrations.AlterField(
            model_name='assignment',
            name='id',
            field=models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID'),
        ),
        migrations.AlterField(
            model_name='billable',
            name='id',
            field=models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID'),
        ),
        migrations.AlterField(
            model_name='billingperiod',
            name='id',
            field=models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID'),
        ),
        migrations.AlterField(
            model_name='delivery',
            name='id',
            field=models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID'),
        ),
        migrations.AlterField(
            model_name='deliveryitem',
            name='id',
            field=models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID'),
        ),
        migrations.AlterField(
            model_name='depot',
            name='id',
            field=models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID'),
        ),
        migrations.AlterField(
            model_name='job',
            name='id',
            field=models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID'),
        ),
        migrations.AlterField(
            model_name='jobextra',
            name='id',
            field=models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID'),
        ),
        migrations.AlterField(
            model_name='jobextratype',
            name='id',
            field=models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID'),
        ),
        migrations.AlterField(
            model_name='jobtype',
            name='id',
            field=models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID'),
        ),
        migrations.AlterField(
            model_name='listmessage',
            name='id',
            field=models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID'),
        ),
        migrations.AlterField(
            model_name='mailtemplate',
            name='id',
            field=models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID'),
        ),
        migrations.AlterField(
            model_name='member',
            name='id',
            field=models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID'),
        ),
        migrations.AlterField(
            model_name='specialroles',
            name='id',
            field=models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID'),
        ),
        migrations.AlterField(
            model_name='subscriptionmembership',
            name='id',
            field=models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID'),
        ),
        migrations.AlterField(
            model_name='subscriptionpart',
            name='id',
            field=models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID'),
        ),
        migrations.AlterField(
            model_name='subscriptionproduct',
            name='id',
            field=models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID'),
        ),
        migrations.AlterField(
            model_name='subscriptionsize',
            name='id',
            field=models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID'),
        ),
        migrations.AlterField(
            model_name='subscriptiontype',
            name='id',
            field=models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID'),
        ),
    ]

[greemo]

Hmm, looks like the normal foreign key relationships may be getting updated, and this has to do with the use of django polymorphic.

It seems somehow all the hidden foreign keys here are not getting updated.

class JuntagricoBasePoly(PolymorphicModel):

    class Meta:
        abstract = True

class Billable(JuntagricoBasePoly):
    ...

class Subscription(Billable, SimpleStateModel):
    ...

class SubscriptionMembership(JuntagricoBaseModel):
    member = models.ForeignKey('Member', on_delete=models.CASCADE)
    subscription = models.ForeignKey('Subscription', db_constraint=False, on_delete=models.CASCADE)
    ...

Error received when running migration

MySQLdb._exceptions.OperationalError: (1833, "Cannot change column 'billable_ptr_id': used in a foreign key constraint 'juntagrico_subscript_subscription_id_1f724c85_fk_juntagric' of table 'juntagrico.juntagrico_subscriptionmembership'")

}}}

[Mariusz Felisiak]

Thanks for the report, however I cannot reproduce this issue without django-polymorphic, i.e. with

class JuntagricoBasePoly(models.Model):

Moreover django-polymorphic doesn't officially support Django 3.2. Please reopen the ticket if you can provide details about why and where Django is at fault, or a sample project that reproduces this issue (without django-polymorphic).

[Dan Dyer]

I've just encountered the same error migrating a project from Django 2.2.24 to 3.2.7. I am not using django-polymorphic. In my case the foreign key that was causing the problem was a reference back to the same model (e.g. parent = models.ForeignKey('self')).

I am using MySQL.
I had set DEFAULT_AUTO_FIELD = 'django.db.models.BigAutoField' in settings.py
The migration failed with a warning about truncation.

I just switched to using AutoField since my existing columns were that size already, but there does appear to be a problem to be fixed.

[Mariusz Felisiak]

Replying to Dan Dyer:

I've just encountered the same error migrating a project from Django 2.2.24 to 3.2.7. I am not using django-polymorphic. In my case the foreign key that was causing the problem was a reference back to the same model (e.g. parent = models.ForeignKey('self')).

I am using MySQL.
I had set DEFAULT_AUTO_FIELD = 'django.db.models.BigAutoField' in settings.py
The migration failed with a warning about truncation.

I just switched to using AutoField since my existing columns were that size already, but there does appear to be a problem to be fixed.

Thanks, however it seems to be a different issue. The original report doesn't mention ForeignKey('self'), uses django-polymorphic, and describes an error not warning. Please open a separate ticket with a sample project to reproduce or a minimal set of models.