PostgreSQL removes constraints when removing columns.

[George]

I’ve encountered an odd bug(?) in Django in our production code. Dropping a column manually in migrations *without* removing its constraints leads to Django being unaware those constraints have been removed, and auto-generating incorrect migrations.

A short example:

Creating the migrations for this schema:

class M():
    a = models.IntegerField(default=1)
    b = models.IntegerField(default=2)
    class Meta:
        constraints = [
            UniqueConstraint(
                name="uniq_ab_1”, fields=[“a”, “b”]
            )
        ]

Create the constraint uniq_ab_1 is Postgres as expected (when using the \d+ command on the table to visualize)

However, this manual migration will remove the constraint, due to one of its member columns being deleted, this is just standard Postgres behavior:

        migrations.RemoveField(
            model_name=“m”,
            name=“a”,
        ),
        migrations.AddField(
            model_name=“m”,
            name=“a”,
            field=models.IntegerField(default=6),
        ),

This migration runs just fine. I can even modify the field of M again and run a further migration. However, using \d+ reveals that the uniq_ab_1 constraint` is gone from the database.

The only way I found out about this behavior was by renaming the constraint to uniq_ab_2, then auto-generating migrations and getting the error:
django.db.utils.ProgrammingError: constraint "uniq_ab_1" of relation … does not exist. In other words, on a rename, Django become aware a rename was happening and tried to remove the constraint from the database, in spite of itt being gone for a few migrations.

This behavior is rather unexpected. I’d assume Django would either:

1. Notice the code model differs from the database schema in the original migration (when the constraint gets inadvertently removed) and fail
2. Notice that the constraint is missing in the next migration (e.g. when adding an arbitrary field to M) and try to add it back again.

As it stands it seems like this ghost constraint is observed by some migration operations and not others.

Is this a known bug in Django?
Is there a way to guard against this behavior?
Is this perfectly normal and I am doing something wrong?

[Mariusz Felisiak]

Thanks for the report, however PostgreSQL silently drops constraints, not Django itself. Unfortunately when you create migrations manually you must be aware of database-specific behavior and handle them on your own in most of cases. As far as I'm aware described use case (removing column to add it immediately in the same migration) is really unusual and not worth additional complexity and a performance regression for all users. Noticing that constraint is missing would require inspecting db constraints which is always quite expensive.

Is there any reason you cannot handle this automatically with AlterField()?

[George]

Replying to Mariusz Felisiak:

Thanks for the report, however PostgreSQL silently drops constraints, not Django itself. Unfortunately when you create migrations manually you must be aware of database-specific behavior and handle them on your own in most of cases. As far as I'm aware described use case (removing column to add it immediately in the same migration) is really unusual and not worth additional complexity and a performance regression for all users. Noticing that constraint is missing would require inspecting db constraints which is always quite expensive.

Is there any reason you cannot handle this automatically with AlterField()?

No particular reason, no, and in this case I agree the issue is we had a poorly written manual migration, fault is on us. I just wanted to make sure this is indeed the behavior.

Out of curiosity, does django inspect the database at all for migrations or is it 100% looking at the mdoel's code and comparing it with the code from all previous migrations?

[Mariusz Felisiak]

Replying to George:

Out of curiosity, does django inspect the database at all for migrations or is it 100% looking at the mdoel's code and comparing it with the code from all previous migrations?

Django migrations inspects database in many places, but all of them slow things down and this scenario is so specific that IMO it's not worth additional guard.