Migrations for fields with model-referencing defaults break later

[Kenny Loveall]

Use case/Goal: I want to generate short (4-8 character), unique codes for orders on a website (think confirmation numbers of airlines, car license plates or the like). With sufficient length, I could just UUIDs and be reasonably guaranteed that I won't see a collision in my lifetime, with the smaller dataspace to pick from, this becomes no longer true. Therefore, I generate a new random one, check to see if it's already in use, and if so, try another one until I find one that works (and log a warning on a first collision and error out if I try too many).

Attempted code:
models.py:

def generate_order_public_reference_number():
        attempts = 0
        while True:
            candidate_str = 'o' + uuid.uuid4().hex[:5]
            try:
                Order.objects.get(public_reference_number=candidate_str)
            except Order.DoesNotExist:
                return candidate_str
            attempts += 1
            if attempts > 10:
                raise RuntimeError("Unable to find an unused public_reference_number after 10 attempts.")

class Order(models.Model):
    ...
    public_reference_number = models.CharField(max_length=10,
                                               null=False,
                                               default=generate_order_public_reference_number)

migration file:

def forwards(apps, schema_editor):
    Order = apps.get_model('orders', 'Order')
    for o in Order.objects.all():
        while True:
            candidate_str = 'o' + uuid.uuid4().hex[:6]
            try:
                Order.objects.get(public_reference_number=candidate_str)
            except Order.DoesNotExist:
                o.public_reference_number = candidate_str
                o.save()
                break


class Migration(migrations.Migration):
    dependencies = [
        ('orders', '0005_auto_20221120_0218'),
    ]

    operations = [
        migrations.AddField(
            model_name='order',
            name='public_reference_number',
            field=models.CharField(null=True, default=None, max_length=10),
        ),
        migrations.RunPython(forwards, migrations.RunPython.noop),
        migrations.AlterField(
            model_name='order',
            name='public_reference_number',
            field=models.CharField(default=orders.models.generate_order_public_reference_number, max_length=10), ),
    ]

What happens: After adding a new field in the future to Order, I get an exception when attempting to apply this migration because the migration engine calls the default callable in the migration to get the default to set on the rows even though I have guaranteed in the row above that there are no rows that will need that default.

What I expect to happen: Only try to call the default function if there's a need for the default value (or even the potential for a need). In this case, the default was already set on the model so we know for sure there's no rows that will need a default filled in.

[Mariusz Felisiak]

Replying to Kenny Loveall:

Use case/Goal: I want to generate short (4-8 character), unique codes for orders on a website (think confirmation numbers of airlines, car license plates or the like). With sufficient length, I could just UUIDs and be reasonably guaranteed that I won't see a collision in my lifetime, with the smaller dataspace to pick from, this becomes no longer true. Therefore, I generate a new random one, check to see if it's already in use, and if so, try another one until I find one that works (and log a warning on a first collision and error out if I try too many).

Have you tried to do this in a separate migration files (as described in ​docs for adding unique fields)?

What happens: After adding a new field in the future to Order, I get an exception when attempting to apply this migration because the migration engine calls the default callable in the migration to get the default to set on the rows even though I have guaranteed in the row above that there are no rows that will need that default.

I cannot reproduce an error when running separate migration files. What kind of exception do you encounter?

What I expect to happen: Only try to call the default function if there's a need for the default value (or even the potential for a need). In this case, the default was already set on the model so we know for sure there's no rows that will need a default filled in.

Django cannot know and evaluate manual operations as RunPython or RunSQL.