Cannot save record with UUID field after migrating existing UUIDField created in Django < 5.0

[Emanuel Andrecut]

Django 5.0 release notes specifies that on MariaDB 10.7+, UUIDField is now created as UUID column rather than CHAR(32) column and migrations where "char(32)" is hardcoded must be added to keep compatibility for UUIDField<s> created before version 5.0: ​https://docs.djangoproject.com/en/5.0/releases/5.0/#migrating-existing-uuidfield-on-mariadb-10-7

However after applying such migrations as described in the release notes records cannot be saved anymore:

  File "/home/manu/venv/lib/python3.11/site-packages/django/db/models/query.py", line 953, in get_or_create
    return self.create(**params), True
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/manu/venv/lib/python3.11/site-packages/django/db/models/query.py", line 677, in create
    obj.save(force_insert=True, using=self.db)
  File "/home/manu/venv/lib/python3.11/site-packages/django/db/models/base.py", line 814, in save
    self.save_base(
  File "/home/manu/venv/lib/python3.11/site-packages/django/db/models/base.py", line 901, in save_base
    updated = self._save_table(
              ^^^^^^^^^^^^^^^^^
  File "/home/manu/venv/lib/python3.11/site-packages/django/db/models/base.py", line 1059, in _save_table
    results = self._do_insert(
              ^^^^^^^^^^^^^^^^
  File "/home/manu/venv/lib/python3.11/site-packages/django/db/models/base.py", line 1100, in _do_insert
    return manager._insert(
           ^^^^^^^^^^^^^^^^
  File "/home/manu/venv/lib/python3.11/site-packages/django/db/models/manager.py", line 87, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/manu/venv/lib/python3.11/site-packages/django/db/models/query.py", line 1845, in _insert
    return query.get_compiler(using=using).execute_sql(returning_fields)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/manu/venv/lib/python3.11/site-packages/django/db/models/sql/compiler.py", line 1822, in execute_sql
    cursor.execute(sql, params)
  File "/home/manu/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 79, in execute
    return self._execute_with_wrappers(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/manu/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 92, in _execute_with_wrappers
    return executor(sql, params, many, context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/manu/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 100, in _execute
    with self.db.wrap_database_errors:
  File "/home/manu/venv/lib/python3.11/site-packages/django/db/utils.py", line 91, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/home/manu/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 105, in _execute
    return self.cursor.execute(sql, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/manu/venv/lib/python3.11/site-packages/django/db/backends/mysql/base.py", line 75, in execute
    return self.cursor.execute(query, args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/manu/venv/lib/python3.11/site-packages/MySQLdb/cursors.py", line 179, in execute
    res = self._query(mogrified_query)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/manu/venv/lib/python3.11/site-packages/MySQLdb/cursors.py", line 330, in _query
    db.query(q)
  File "/home/manu/venv/lib/python3.11/site-packages/MySQLdb/connections.py", line 255, in query
    _mysql.connection.query(self, query)
django.db.utils.DataError: (1406, "Data too long for column 'id' at row 1")

That is because, before this commit ​https://github.com/django/django/commit/7cd187a5ba58d7769039f487faeb9a5a2ff05540, has_native_uuid_field would return False and after the 5.0 change, saving an UUID for old fields that must still use CHAR(32) will not try to save it using the hex value of UUID: ​https://github.com/django/django/blob/main/django/db/models/fields/__init__.py#L2737

[Emanuel Andrecut]

I've included a possible fix here: ​https://github.com/django/django/pull/17604/commits/432d4a28b7c79db18f79f091bd66d428ae606483

[Mariusz Felisiak]

Thanks for the report! Sounds reasonable, I'm not sure if we want to add workarounds to the Django itself. What do you think about updating release notes? For example:

docs/releases/5.0.txt

@@ -508 +508 @@
         def db_type(self, connection):
             return "char(32)"
+
+        def get_db_prep_value(self, value, connection, prepared=False):
+            value = super().get_db_prep_value(value, connection, prepared)
+            if value is not None:
+                value = value.hex
+            return value
+
 For example::
 
     class MyModel(models.Model):
@@ -516 +523 @@
 Should become::
 
     class Char32UUIDField(models.UUIDField):
-        def db_type(self, connection):
-            return "char(32)"
+        ...
 
 
     class MyModel(models.Model):

Regression in 7cd187a5ba58d7769039f487faeb9a5a2ff05540.

[Emanuel Andrecut]

I've dropped the Django changes in the PR and updated class Char32UUIDField in release notes, please check.

Replying to Mariusz Felisiak:

Thanks for the report! Sounds reasonable, I'm not sure if we want to add workarounds to the Django itself. What do you think about updating release notes? For example:

docs/releases/5.0.txt

@@ -508 +508 @@
         def db_type(self, connection):
             return "char(32)"
+
+        def get_db_prep_value(self, value, connection, prepared=False):
+            value = super().get_db_prep_value(value, connection, prepared)
+            if value is not None:
+                value = value.hex
+            return value
+
 For example::
 
     class MyModel(models.Model):
@@ -516 +523 @@
 Should become::
 
     class Char32UUIDField(models.UUIDField):
-        def db_type(self, connection):
-            return "char(32)"
+        ...
 
 
     class MyModel(models.Model):

Regression in 7cd187a5ba58d7769039f487faeb9a5a2ff05540.

[Mariusz Felisiak]

Replying to Emanuel Andrecut:

I've dropped the Django changes in the PR and updated class Char32UUIDField in release notes, please check.

Looks good, thanks.

[Mariusz Felisiak <felisiak.mariusz@…>]

In e72b282:

Fixed #35032 -- Corrected Char32UUIDField implementation in 5.0 release notes.

This fixes Char32UUIDField implementation in 5.0 release notes causing
records with UUIDFields created using pre-Django 5.0 and CHAR(32) not
being able to be saved anymore after upgrading and keeping the CHAR(32)
columns.

Regression in 7cd187a5ba58d7769039f487faeb9a5a2ff05540.

[Mariusz Felisiak <felisiak.mariusz@…>]

In 636d701:

[5.0.x] Fixed #35032 -- Corrected Char32UUIDField implementation in 5.0 release notes.

This fixes Char32UUIDField implementation in 5.0 release notes causing
records with UUIDFields created using pre-Django 5.0 and CHAR(32) not
being able to be saved anymore after upgrading and keeping the CHAR(32)
columns.

Regression in 7cd187a5ba58d7769039f487faeb9a5a2ff05540.

Backport of e72b2826ff1eaf2f48ee54a40d2f2988a1fdbb0a from main