Missing or misinferred attributes in output fields of generated fields

[Paolo Melchiorre]

Continuing with my experiments with the generated fields I found these two error situations which I reported together because they are very connected and could lead to a single solution.

I do not rule out that there are other examples of use of the generated fields that could lead to the same errors and if they come to mind, please add them in the comments.

Misinferred attributes

In the common example of a field generated as a concatenation of two CharField in a Model:

class Person(models.Model):
    first_name = models.CharField(max_length=255)
    last_name = models.CharField(max_length=255)
    full_name = models.GeneratedField(
        expression=Concat(
            "first_name", models.Value(" "), "last_name"
        ),
        db_persist=True,
    )

The SQL code for the generated column has an automatically inferred max length of only 255 characters (varchar(255)), while the field should be able to contain strings of 511 characters (varchar(511)):

CREATE TABLE "community_person" (
    "id" integer NOT NULL PRIMARY KEY AUTOINCREMENT,
    "first_name" varchar(255) NOT NULL,
    "last_name" varchar(255) NOT NULL,
    "full_name" varchar(255) GENERATED ALWAYS AS (
        COALESCE("first_name", '') ||
        COALESCE(COALESCE(' ', '') ||
        COALESCE("last_name", ''), '')
    ) STORED
);

Proposal

To solve the problem you could alternatively:

1. make the maximum length extraction process smarter for the automatically created output fields
2. mandatorily require specifying the output field when some of the fields involved could lead to situations like the ones above

Missing attributes

If in the previous example, I explicitly specify the output field without attributes, the migration is generated without generating errors:

class Person(models.Model):
    first_name = models.CharField(max_length=255)
    last_name = models.CharField(max_length=255)
    full_name = models.GeneratedField(
        expression=Concat(
            "first_name", models.Value(" "), "last_name"
        ),
        db_persist=True,
        output_field=models.CharField(),
    )

The SQL code contains an incorrect varchar(None) type:

CREATE TABLE "community_person" (
    "id" integer NOT NULL PRIMARY KEY AUTOINCREMENT,
    "first_name" varchar(255) NOT NULL,
    "last_name" varchar(255) NOT NULL,
    "full_name" varchar(None) GENERATED ALWAYS AS (
        COALESCE("first_name", '') ||
        COALESCE(COALESCE(' ', '') ||
        COALESCE("last_name", ''), '')
    ) STORED
);

which will generate an error when trying to apply the migration to the database:

sqlite3.OperationalError: near "None": syntax error

Proposal

To solve the problem you could alternatively:

1. make the SQL code generation process smarter so as to generate working SQL code
2. request to specify mandatory attributes for the output fields raising an error in the migration creation phase

Cumulative proposal

To solve both problems shown above, given the little time remaining before the release of the stable version of Django 5, the two most drastic solutions from both of the above proposals could be adopted.

Required output field

Always require specifying the output field (except when you are sure that the extracted type cannot generate error situations?)

Example of error message:

django.core.exceptions.FieldError: Expression doesn't contain an explicit type. You must set output_field.

Required attributes

Request to specify mandatory attributes for the output fields raising an error in the migration creation phase.

$ python3 -m manage makemigrations
SystemCheckError: System check identified some issues:

ERRORS:
community.Person.full_name: (fields.E120) CharFields must define a 'max_length' attribute.