Migrations raise AppRegistryNotReady when GeneratedField references incorrect fields.

[Paolo Melchiorre]

Note: In my experiments, I am sure I have encountered similar errors even in cases that are simpler to reproduce, without models with multi-table inheritance to clarify, but in this case, I cannot find the example code of the field generated to reproduce it, so I report below the example that certainly allows you to reproduce the error.

---

I tried to test the behavior of the generated fields in the case of multi-table inheritance.

I was expecting an error since the documentation specifies that:

The expressions should be deterministic and only reference fields within the model (in the same database table).

The generation of migrations of two inheritance models occurs successfully (without generated fields)

from django.db import models

class Person(models.Model):
    first_name = models.CharField(max_length=150)
    last_name = models.CharField(max_length=150)

class Profile(Person):
    email = models.EmailField()

$ python3 -m manage makemigrations
Migrations for 'samples':
  samples/migrations/0010_person_profile.py
    - Create model Person
    - Create model Profile
$ python3 -m manage sqlmigrate samples 0010

BEGIN;
--
-- Create model Person
--
CREATE TABLE "samples_person" ("id" integer NOT NULL PRIMARY KEY AUTOINCREMENT, "first_name" varchar(150) NOT NULL, "last_name" varchar(150) NOT NULL);
--
-- Create model Profile
--
CREATE TABLE "samples_profile" ("person_ptr_id" bigint NOT NULL PRIMARY KEY REFERENCES "samples_person" ("id") DEFERRABLE INITIALLY DEFERRED, "email" varchar(254) NOT NULL);
COMMIT;

Instead, if in the second model, I create a generated field that references the fields in the first model I get a very unclear error that perhaps we could make more clear.

from django.db import models
from django.db.models import Value
from django.db.models.functions import (
    Concat,
)

class Person(models.Model):
    first_name = models.CharField(max_length=150)
    last_name = models.CharField(max_length=150)

class Profile(Person):
    email = models.EmailField()
    display_name = models.GeneratedField(
        expression=Concat("first_name", Value(" "), "last_name", Value(" "), "email",),
        db_persist=True,
        output_field=models.TextField(),
    )

$ python3 -m manage makemigrations

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/home/paulox/Projects/generatedfields/manage.py", line 22, in <module>
    main()
  File "/home/paulox/Projects/generatedfields/manage.py", line 18, in main
    execute_from_command_line(sys.argv)
  File "/home/paulox/.virtualenvs/generatedfields/lib/python3.12/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
    utility.execute()
  File "/home/paulox/.virtualenvs/generatedfields/lib/python3.12/site-packages/django/core/management/__init__.py", line 416, in execute
    django.setup()
  File "/home/paulox/.virtualenvs/generatedfields/lib/python3.12/site-packages/django/__init__.py", line 24, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/home/paulox/.virtualenvs/generatedfields/lib/python3.12/site-packages/django/apps/registry.py", line 116, in populate
    app_config.import_models()
  File "/home/paulox/.virtualenvs/generatedfields/lib/python3.12/site-packages/django/apps/config.py", line 269, in import_models
    self.models_module = import_module(models_module_name)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/importlib/__init__.py", line 90, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1381, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1354, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1325, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 929, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 994, in exec_module
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "/home/paulox/Projects/generatedfields/samples/models.py", line 233, in <module>
    class Profile(User):
  File "/home/paulox/.virtualenvs/generatedfields/lib/python3.12/site-packages/django/db/models/base.py", line 194, in __new__
    new_class.add_to_class(obj_name, obj)
  File "/home/paulox/.virtualenvs/generatedfields/lib/python3.12/site-packages/django/db/models/base.py", line 371, in add_to_class
    value.contribute_to_class(cls, name)
  File "/home/paulox/.virtualenvs/generatedfields/lib/python3.12/site-packages/django/db/models/fields/generated.py", line 51, in contribute_to_class
    self._resolved_expression = self.expression.resolve_expression(
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/paulox/.virtualenvs/generatedfields/lib/python3.12/site-packages/django/db/models/expressions.py", line 975, in resolve_expression
    c.source_expressions[pos] = arg.resolve_expression(
                                ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/paulox/.virtualenvs/generatedfields/lib/python3.12/site-packages/django/db/models/expressions.py", line 975, in resolve_expression
    c.source_expressions[pos] = arg.resolve_expression(
                                ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/paulox/.virtualenvs/generatedfields/lib/python3.12/site-packages/django/db/models/expressions.py", line 854, in resolve_expression
    return query.resolve_ref(self.name, allow_joins, reuse, summarize)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/paulox/.virtualenvs/generatedfields/lib/python3.12/site-packages/django/db/models/sql/query.py", line 2001, in resolve_ref
    join_info = self.setup_joins(
                ^^^^^^^^^^^^^^^^^
  File "/home/paulox/.virtualenvs/generatedfields/lib/python3.12/site-packages/django/db/models/sql/query.py", line 1854, in setup_joins
    path, final_field, targets, rest = self.names_to_path(
                                       ^^^^^^^^^^^^^^^^^^^
  File "/home/paulox/.virtualenvs/generatedfields/lib/python3.12/site-packages/django/db/models/sql/query.py", line 1754, in names_to_path
    *get_field_names_from_opts(opts),
     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/paulox/.virtualenvs/generatedfields/lib/python3.12/site-packages/django/db/models/sql/query.py", line 63, in get_field_names_from_opts
    (f.name, f.attname) if f.concrete else (f.name,) for f in opts.get_fields()
                                                              ^^^^^^^^^^^^^^^^^
  File "/home/paulox/.virtualenvs/generatedfields/lib/python3.12/site-packages/django/db/models/options.py", line 858, in get_fields
    return self._get_fields(
           ^^^^^^^^^^^^^^^^^
  File "/home/paulox/.virtualenvs/generatedfields/lib/python3.12/site-packages/django/db/models/options.py", line 931, in _get_fields
    all_fields = self._relation_tree
                 ^^^^^^^^^^^^^^^^^^^
  File "/home/paulox/.virtualenvs/generatedfields/lib/python3.12/site-packages/django/utils/functional.py", line 47, in __get__
    res = instance.__dict__[self.name] = self.func(instance)
                                         ^^^^^^^^^^^^^^^^^^^
  File "/home/paulox/.virtualenvs/generatedfields/lib/python3.12/site-packages/django/db/models/options.py", line 831, in _relation_tree
    return self._populate_directed_relation_graph()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/paulox/.virtualenvs/generatedfields/lib/python3.12/site-packages/django/db/models/options.py", line 798, in _populate_directed_relation_graph
    all_models = self.apps.get_models(include_auto_created=True)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/paulox/.virtualenvs/generatedfields/lib/python3.12/site-packages/django/apps/registry.py", line 181, in get_models
    self.check_models_ready()
  File "/home/paulox/.virtualenvs/generatedfields/lib/python3.12/site-packages/django/apps/registry.py", line 143, in check_models_ready
    raise AppRegistryNotReady("Models aren't loaded yet.")
django.core.exceptions.AppRegistryNotReady: Models aren't loaded yet.

[Mariusz Felisiak]

Thanks for the report. We could add a small check for GeneratedField references, e.g.

django/db/models/base.py

@@ -1601 +1601 @@
                 *cls._check_constraints(databases),
                 *cls._check_default_pk(),
                 *cls._check_db_table_comment(databases),
+                *cls._check_generated_fields(databases),
             ]
 
         return errors
@@ -1957 +1958 @@
                 errors.extend(cls._check_local_fields(fields, "unique_together"))
             return errors
 
+    @classmethod
+    def _check_generated_fields(cls, databases):
+        errors = []
+        for f in cls._meta.local_fields:
+            if not f.generated:
+                continue
+            references = {
+                ref[0] for ref in cls._get_expr_references(f.expression)
+            }
+            errors.extend(cls._check_local_fields(references, f.name))
+        return errors
+
     @classmethod
     def _check_indexes(cls, databases):
         """Check fields, names, and conditions of indexes."""

Unfortunately, this would still required catching exceptions when resolving GeneratedField expressions.

[Mariusz Felisiak]

Unfortunately, this would still required catching exceptions when resolving GeneratedField expressions.

Or we can move resolving an expression to the generated_sql() which is not the end of the world:

django/db/models/fields/generated.py

@@ -13 +13 @@
     db_returning = True
 
     _query = None
-    _resolved_expression = None
     output_field = None
 
     def __init__(self, *, expression, output_field, db_persist=None, **kwargs):
@@ -48 +47 @@
         super().contribute_to_class(*args, **kwargs)
 
         self._query = Query(model=self.model, alias_cols=False)
-        self._resolved_expression = self.expression.resolve_expression(
-            self._query, allow_joins=False
-        )
         # Register lookups from the output_field class.
         for lookup_name, lookup in self.output_field.get_class_lookups().items():
             self.register_lookup(lookup, lookup_name=lookup_name)
@@ -59 +55 @@
         compiler = connection.ops.compiler("SQLCompiler")(
             self._query, connection=connection, using=None
         )
-        return compiler.compile(self._resolved_expression)
+        resolved_expression = self.expression.resolve_expression(
+            self._query, allow_joins=False
+        )
+        return compiler.compile(resolved_expression)
 
     def check(self, **kwargs):
         databases = kwargs.get("databases") or []

[Mariusz Felisiak]

On reconsideration, system check is not necessary. makemigrations doesn't crash with the proposed diff, and migrate raises:

django.core.exceptions.FieldError: Joined field references are not permitted in this query

which is fine, IMO.

[Mariusz Felisiak]

​PR

[GitHub <noreply@…>]

In 101a85a:

Fixed #34985 -- Fixed GeneratedFields.contribute_to_class() crash when apps are not populated.

Thanks Paolo Melchiorre for the report.

Regression in f333e3513e8bdf5ffeb6eeb63021c230082e6f95.

[Mariusz Felisiak <felisiak.mariusz@…>]

In 48eebdc:

[5.0.x] Fixed #34985 -- Fixed GeneratedFields.contribute_to_class() crash when apps are not populated.

Thanks Paolo Melchiorre for the report.

Regression in f333e3513e8bdf5ffeb6eeb63021c230082e6f95.
Backport of 101a85a5a06585ba16ecb25860146d034a8a55ec from main