Migration with CheckConstraint fails on PostgreSQL in PyPy with psycopg2cffi due to AttributeError

[Henryk Plötz]

Description

quote_value in django/db/backends/postgresql/schema.py assigns to adapted.encoding. This is not allowed in psycopg2cffi and raises an AttributeError.

The code path was added in #31815

I'm not sure what the correct fix would be. Simply not assigning encoding seems wrong. Is what the code tries even possible in cffi?

Steps to reproduce

(I'm using poetry. Also needs apt-get install libpq-dev pypy3 pypy3-dev or equivalent.)

poetry new demo
cd demo
poetry env use `which pypy3`
poetry add django psycopg2cffi
rm -rf demo
poetry run django-admin startproject demo
cd demo
poetry run ./manage.py startapp demo1

in demo/settings.py add

from psycopg2cffi import compat
compat.register()

and add demo1 to INSTALLED_APPS

in demo1/models.py add

from django.db import models

# Create your models here.

class Demo1(models.Model):
  dummy = models.CharField(max_length=23)
  class Meta:
    constraints = [
      models.CheckConstraint(name="dummy", check=models.Q(dummy__in=["a","b"]))
    ]

then run

poetry run ./manage.py makemigrations

The file demo1/migrations/0001_initial.py now contains

# Generated by Django 4.1.3 on 2022-11-02 10:25

from django.db import migrations, models


class Migration(migrations.Migration):

    initial = True

    dependencies = [
    ]

    operations = [
        migrations.CreateModel(
            name='Demo1',
            fields=[
                ('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
                ('dummy', models.CharField(max_length=23)),
            ],
        ),
        migrations.AddConstraint(
            model_name='demo1',
            constraint=models.CheckConstraint(check=models.Q(('dummy__in', ['a', 'b'])), name='dummy'),
        ),
    ]

Now configure settings for a postgres database (in demo/settings.py)

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql',
        'NAME': 'demo',
        "USER": "demo",
        "PASSWORD": "XXXX",
        "HOST": "XXXX",
    }
}

and run

poetry run ./manage.py migrate

Actual results

Operations to perform:
  Apply all migrations: admin, auth, contenttypes, demo1, sessions
Running migrations:
  Applying contenttypes.0001_initial... OK
  Applying auth.0001_initial... OK
  Applying admin.0001_initial... OK
  Applying admin.0002_logentry_remove_auto_add... OK
  Applying admin.0003_logentry_add_action_flag_choices... OK
  Applying contenttypes.0002_remove_content_type_name... OK
  Applying auth.0002_alter_permission_name_max_length... OK
  Applying auth.0003_alter_user_email_max_length... OK
  Applying auth.0004_alter_user_username_opts... OK
  Applying auth.0005_alter_user_last_login_null... OK
  Applying auth.0006_require_contenttypes_0002... OK
  Applying auth.0007_alter_validators_add_error_messages... OK
  Applying auth.0008_alter_user_username_max_length... OK
  Applying auth.0009_alter_user_last_name_max_length... OK
  Applying auth.0010_alter_group_name_max_length... OK
  Applying auth.0011_update_proxy_permissions... OK
  Applying auth.0012_alter_user_first_name_max_length... OK
  Applying demo1.0001_initial...Traceback (most recent call last):
  File "./manage.py", line 22, in <module>
    main()
  File "./manage.py", line 18, in main
    execute_from_command_line(sys.argv)
  File "/home/henryk/.cache/pypoetry/virtualenvs/demo-gWrR8tCo-py3.8/lib/pypy3.8/site-packages/django/core/management/__init__.py", line 446, in execute_from_command_line
    utility.execute()
  File "/home/henryk/.cache/pypoetry/virtualenvs/demo-gWrR8tCo-py3.8/lib/pypy3.8/site-packages/django/core/management/__init__.py", line 440, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/henryk/.cache/pypoetry/virtualenvs/demo-gWrR8tCo-py3.8/lib/pypy3.8/site-packages/django/core/management/base.py", line 402, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/home/henryk/.cache/pypoetry/virtualenvs/demo-gWrR8tCo-py3.8/lib/pypy3.8/site-packages/django/core/management/base.py", line 448, in execute
    output = self.handle(*args, **options)
  File "/home/henryk/.cache/pypoetry/virtualenvs/demo-gWrR8tCo-py3.8/lib/pypy3.8/site-packages/django/core/management/base.py", line 96, in wrapped
    res = handle_func(*args, **kwargs)
  File "/home/henryk/.cache/pypoetry/virtualenvs/demo-gWrR8tCo-py3.8/lib/pypy3.8/site-packages/django/core/management/commands/migrate.py", line 354, in handle
    fake_initial=fake_initial,
  File "/home/henryk/.cache/pypoetry/virtualenvs/demo-gWrR8tCo-py3.8/lib/pypy3.8/site-packages/django/db/migrations/executor.py", line 136, in migrate
    state, plan, full_plan, fake=fake, fake_initial=fake_initial
  File "/home/henryk/.cache/pypoetry/virtualenvs/demo-gWrR8tCo-py3.8/lib/pypy3.8/site-packages/django/db/migrations/executor.py", line 168, in _migrate_all_forwards
    state, migration, fake=fake, fake_initial=fake_initial
  File "/home/henryk/.cache/pypoetry/virtualenvs/demo-gWrR8tCo-py3.8/lib/pypy3.8/site-packages/django/db/migrations/executor.py", line 252, in apply_migration
    state = migration.apply(state, schema_editor)
  File "/home/henryk/.cache/pypoetry/virtualenvs/demo-gWrR8tCo-py3.8/lib/pypy3.8/site-packages/django/db/migrations/migration.py", line 131, in apply
    self.app_label, schema_editor, old_state, project_state
  File "/home/henryk/.cache/pypoetry/virtualenvs/demo-gWrR8tCo-py3.8/lib/pypy3.8/site-packages/django/db/migrations/operations/models.py", line 1040, in database_forwards
    schema_editor.add_constraint(model, self.constraint)
  File "/home/henryk/.cache/pypoetry/virtualenvs/demo-gWrR8tCo-py3.8/lib/pypy3.8/site-packages/django/db/backends/base/schema.py", line 511, in add_constraint
    sql = constraint.create_sql(model, self)
  File "/home/henryk/.cache/pypoetry/virtualenvs/demo-gWrR8tCo-py3.8/lib/pypy3.8/site-packages/django/db/models/constraints.py", line 83, in create_sql
    check = self._get_check_sql(model, schema_editor)
  File "/home/henryk/.cache/pypoetry/virtualenvs/demo-gWrR8tCo-py3.8/lib/pypy3.8/site-packages/django/db/models/constraints.py", line 76, in _get_check_sql
    return sql % tuple(schema_editor.quote_value(p) for p in params)
  File "/home/henryk/.cache/pypoetry/virtualenvs/demo-gWrR8tCo-py3.8/lib/pypy3.8/site-packages/django/db/models/constraints.py", line 76, in <genexpr>
    return sql % tuple(schema_editor.quote_value(p) for p in params)
  File "/home/henryk/.cache/pypoetry/virtualenvs/demo-gWrR8tCo-py3.8/lib/pypy3.8/site-packages/django/db/backends/postgresql/schema.py", line 57, in quote_value
    adapted.encoding = "utf8"
AttributeError: can't set attribute

Expected results

No exception :)

[Mariusz Felisiak]

Thanks for the report, however psycopg2cffi is not officially supported. Moreover, it's compatible with psycopg2==2.5 (according to its README) and Django 4.1 supports psycopg2 2.8.4 or higher. You should report this on the psycopg2cffi bug tracker.

[Iuri de Silvio]

Just for reference, I reported it upstream long time ago, there is a patch there but no merge, the project looks abandoned.

​https://github.com/chtd/psycopg2cffi/issues/126