Document required sqlparse version in Django 2.2.

[Fabian Sturm]

When I run a migration during testing with sqlite as database backend I get a backtrace. The reason is a wrong constraint predicate in a list comprehension in introspection.py in line 333:

tokens = (token for token in statement.flatten() if not token.is_whitespace)

it works for me with this change:

tokens = (token for token in statement.flatten() if not token.is_whitespace())

Without this change the tokens list is empty and hence the crash shown below.
This all is on Django 2.2.13 on Ubuntu 18.04 and Python 3.6.9.

backtrace:

  Applying main.0006_auto_20200128_1650...Traceback (most recent call last):
  File "manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "env/lib/python3.6/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
    utility.execute()
  File "env/lib/python3.6/site-packages/django/core/management/__init__.py", line 375, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "env/lib/python3.6/site-packages/django/core/management/commands/test.py", line 23, in run_from_argv
    super().run_from_argv(argv)
  File "env/lib/python3.6/site-packages/django/core/management/base.py", line 323, in run_from_argv
    self.execute(*args, **cmd_options)
  File "env/lib/python3.6/site-packages/django/core/management/base.py", line 364, in execute
    output = self.handle(*args, **options)
  File "env/lib/python3.6/site-packages/django/core/management/commands/test.py", line 53, in handle
    failures = test_runner.run_tests(test_labels)
  File "env/lib/python3.6/site-packages/django/test/runner.py", line 629, in run_tests
    old_config = self.setup_databases(aliases=databases)
  File "env/lib/python3.6/site-packages/django/test/runner.py", line 554, in setup_databases
    self.parallel, **kwargs
  File "env/lib/python3.6/site-packages/django/test/utils.py", line 174, in setup_databases
    serialize=connection.settings_dict.get('TEST', {}).get('SERIALIZE', True),
  File "env/lib/python3.6/site-packages/django/db/backends/base/creation.py", line 72, in create_test_db
    run_syncdb=True,
  File "env/lib/python3.6/site-packages/django/core/management/__init__.py", line 148, in call_command
    return command.execute(*args, **defaults)
  File "env/lib/python3.6/site-packages/django/core/management/base.py", line 364, in execute
    output = self.handle(*args, **options)
  File "env/lib/python3.6/site-packages/django/core/management/base.py", line 83, in wrapped
    res = handle_func(*args, **kwargs)
  File "env/lib/python3.6/site-packages/django/core/management/commands/migrate.py", line 234, in handle
    fake_initial=fake_initial,
  File "env/lib/python3.6/site-packages/django/db/migrations/executor.py", line 117, in migrate
    state = self._migrate_all_forwards(state, plan, full_plan, fake=fake, fake_initial=fake_initial)
  File "env/lib/python3.6/site-packages/django/db/migrations/executor.py", line 147, in _migrate_all_forwards
    state = self.apply_migration(state, migration, fake=fake, fake_initial=fake_initial)
  File "env/lib/python3.6/site-packages/django/db/migrations/executor.py", line 245, in apply_migration
    state = migration.apply(state, schema_editor)
  File "env/lib/python3.6/site-packages/django/db/migrations/migration.py", line 124, in apply
    operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
  File "env/lib/python3.6/site-packages/django/db/migrations/operations/models.py", line 530, in database_forwards
    getattr(new_model._meta, self.option_name, set()),
  File "env/lib/python3.6/site-packages/django/db/backends/base/schema.py", line 363, in alter_unique_together
    self._delete_composed_index(model, fields, {'unique': True}, self.sql_delete_unique)
  File "env/lib/python3.6/site-packages/django/db/backends/base/schema.py", line 391, in _delete_composed_index
    **constraint_kwargs
  File "env/lib/python3.6/site-packages/django/db/backends/base/schema.py", line 1115, in _constraint_names
    constraints = self.connection.introspection.get_constraints(cursor, model._meta.db_table)
  File "env/lib/python3.6/site-packages/django/db/backends/sqlite3/introspection.py", line 375, in get_constraints
    constraints.update(self._parse_table_constraints(table_schema, columns))
  File "env/lib/python3.6/site-packages/django/db/backends/sqlite3/introspection.py", line 353, in _parse_table_constraints
    if end_token.match(sqlparse.tokens.Punctuation, ')'):
AttributeError: 'NoneType' object has no attribute 'match'
Makefile:3: recipe for target 'test' failed
make: *** [test] Error 1

migration:

# -*- coding: utf-8 -*-
from __future__ import unicode_literals

from django.db import models, migrations


class Migration(migrations.Migration):

    dependencies = [
        ('main', '0005_software_deleted'),
    ]

    operations = [
        migrations.AlterField(
            model_name='module',
            name='file_name',
            field=models.CharField(help_text=b'filename under which the module should be saved', max_length=255, editable=False),
            preserve_default=True,
        ),
        migrations.AlterField(
            model_name='testcaseresultdata',
            name='test_case_result',
            field=models.ForeignKey(on_delete=models.CASCADE, related_name='extra_data', to='main.TestCaseResult', unique=True),
            preserve_default=True,
        ),
        migrations.AlterUniqueTogether(
            name='module',
            unique_together=set([]),
        ),
    ]

[Fabian Sturm]

the necessary change is now implemented in this pull request: ​https://github.com/django/django/pull/13039

[Simon Charette]

Mariusz in the light of this issue and ​this comment what do you think of backporting the >=0.2.2 specifier to 2.2.x? It's more of a packaging issue IMO.

[Fabian Sturm]

Yes please backport the version specifier, it cost me a whole day figuring it out. Otherwise how can someone know if Django is running with compatible dependencies?

[Mariusz Felisiak]

Agreed, I missed that we backported breaking change to the Django 2.2. I backported the main change (cdad78e6ee562f8f3acce065e396fb90d04180a1) and prepared a doc fix ​PR.

[GitHub <noreply@…>]

In 4339f2af:

Refs #31682 -- Doc'd minimal sqlparse version in Django 2.2.

Support for sqlparse < 0.2.2 was broken in
40b0a58f5ff949fba1072627e4ad11ef98aa7f36 because is_whitespace property
was added in sqlparse 0.2.2.

[Mariusz Felisiak <felisiak.mariusz@…>]

In 714d7cb:

[3.1.x] Refs #31682 -- Doc'd minimal sqlparse version in Django 2.2.

Support for sqlparse < 0.2.2 was broken in
40b0a58f5ff949fba1072627e4ad11ef98aa7f36 because is_whitespace property
was added in sqlparse 0.2.2.
Backport of 4339f2aff272bceabd67e452c65bcfe0700b3f09 from master

[Mariusz Felisiak <felisiak.mariusz@…>]

In 9ec6eca1:

[3.0.x] Refs #31682 -- Doc'd minimal sqlparse version in Django 2.2.

Support for sqlparse < 0.2.2 was broken in
40b0a58f5ff949fba1072627e4ad11ef98aa7f36 because is_whitespace property
was added in sqlparse 0.2.2.
Backport of 4339f2aff272bceabd67e452c65bcfe0700b3f09 from master

[Mariusz Felisiak <felisiak.mariusz@…>]

In 9ecce34:

[2.2.x] Refs #31682 -- Doc'd minimal sqlparse version in Django 2.2.

Support for sqlparse < 0.2.2 was broken in
40b0a58f5ff949fba1072627e4ad11ef98aa7f36 because is_whitespace property
was added in sqlparse 0.2.2.

Backport of 4339f2aff272bceabd67e452c65bcfe0700b3f09 from master.