Add pgcrypto extension and GEN_RANDOM_UUID function to contrib.postgres

[Paolo Melchiorre]

After the introduction of the UUID Field in Django 1.8, I believe that django.contrib.postgres could benefit from some custom functions based on the pgcrypto extension of PostgreSQL (see ​https://www.postgresql.org/docs/9.6/static/pgcrypto.html). That kind of functions would be very helpful for apply a migration that adds a unique non-nullable field to a table with existing rows.

Starting from "Migrations that add unique fields" (see ​https://docs.djangoproject.com/en/dev/howto/writing-migrations/#migrations-that-add-unique-fields) I speed up the gen_uuid using GEN_RANDOM_UUID function changing it from:

import uuid

def gen_uuid(apps, schema_editor):
    MyModel = apps.get_model('myapp', 'MyModel')
    for row in MyModel.objects.all():
        row.uuid = uuid.uuid4()
        row.save(update_fields=['uuid'])

to

from django.contrib.postgres.functions import RandomUUID

def gen_uuid(apps, schema_editor):
    MyModel = apps.get_model('myapp', 'MyModel')
    MyModel.objects.update(uuid=RandomUUID())

Using this function on my system the time to migrate more than 10000 objects decreased from

real 0m15.988s
user 0m10.680s
sys 0m0.508s

to

real 0m2.957s
user 0m1.736s
sys 0m0.072s

I already implemented a solution for thi feature and I've created a ​PR

[Paolo Melchiorre]

I just submitted a full featured pull request on ghithub ​https://github.com/django/django/pull/8265

[Tim Graham <timograham@…>]

In fcb5dbfe:

Fixed #27996 -- Added RandomUUID function and CryptoExtension to contrib.postgres.

[GitHub <noreply@…>]

In 650bf671:

Refs #27996 -- Skipped RandomUUID test on PostgreSQL 9.3.

[Mariusz Felisiak <felisiak.mariusz@…>]

In 628c4a2:

Refs #27996 -- Doc'd no extension required for RandomUUID() on PostgreSQL 13+.

​https://www.postgresql.org/docs/13/functions-uuid.html
​https://www.postgresql.org/docs/13/pgcrypto.html#id-1.11.7.34.10.5

[Iuri de Silvio]

Hi! Today I updated my master and test migrations failed because of ​https://github.com/django/django/pull/13241 . You checked for is_postgresql_13 even if the connection is not postgresql. It fails with psycopg2 installed but using another db (sqlite in my case).

[Iuri de Silvio]

For completeness, here is the stacktrace.

Traceback (most recent call last):
  File "runtests.py", line 571, in <module>
    failures = django_tests(
  File "runtests.py", line 313, in django_tests
    failures = test_runner.run_tests(
  File "django/django/test/runner.py", line 707, in run_tests
    old_config = self.setup_databases(aliases=databases)
  File "django/django/test/runner.py", line 626, in setup_databases
    return _setup_databases(
  File "django/django/test/utils.py", line 170, in setup_databases
    connection.creation.create_test_db(
  File "django/django/db/backends/base/creation.py", line 65, in create_test_db
    call_command(
  File "django/django/core/management/__init__.py", line 168, in call_command
    return command.execute(*args, **defaults)
  File "django/django/core/management/base.py", line 394, in execute
    output = self.handle(*args, **options)
  File "django/django/core/management/base.py", line 89, in wrapped
    res = handle_func(*args, **kwargs)
  File "django/django/core/management/commands/migrate.py", line 92, in handle
    executor = MigrationExecutor(connection, self.migration_progress_callback)
  File "django/django/db/migrations/executor.py", line 18, in __init__
    self.loader = MigrationLoader(self.connection)
  File "django/django/db/migrations/loader.py", line 53, in __init__
    self.build_graph()
  File "django/django/db/migrations/loader.py", line 210, in build_graph
    self.load_disk()
  File "django/django/db/migrations/loader.py", line 112, in load_disk
    migration_module = import_module(migration_path)
  File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 783, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "django/tests/postgres_tests/migrations/0001_setup_extensions.py", line 22, in <module>
    needs_crypto_extension = not connection.features.is_postgresql_13
AttributeError: 'DatabaseFeatures' object has no attribute 'is_postgresql_13'

[Mariusz Felisiak]

Iuri, thanks for the report, fix is ready ​PR.

[GitHub <noreply@…>]

In 88af11c:

Refs #27996 -- Fixed postgres_tests crash if not running with PostgreSQL.