Spatialite tests could use or create real database

[Andriy Sokolovskiy]

Modify your test config to be like this:

DATABASES = {
    'default': {
        'ENGINE': 'django.contrib.gis.db.backends.spatialite',
        'NAME': 'nonmemory',
        'TEST_NAME': ':memory:',
    },
    'other': {
        'ENGINE': 'django.contrib.gis.db.backends.spatialite',
        'NAME': 'nonmemory2',
        'TEST_NAME': ':memory:',
    }
}

Then run python runtests.py django.contrib.gis.tests.test_geoforms.
File with name nonmemory will be created, it is not correct.

More details about investigation:

Issue was found here:
​https://github.com/django/django/pull/3677#issuecomment-68381272

Some investigation logs:

<truecoldmind> timograham, It is strange. Look at https://github.com/django/django/blob/51890ce8898f821d28f2f6fb6071c936e9bd88f0/django/contrib/gis/tests/utils.py#L39
<truecoldmind> This import causing problems.
<truecoldmind> For example if it is imported, python runtests.py backends django.contrib.gis.tests.test_geoforms will fail, but when I removed it (this import is not used in test_geoforms), tests passing correctly. Any ideas what it could be?
...
<truecoldmind> timograham, in this file connection.ops.spatial_version[0] used, which is performing some database operation. I will try to figure out what is the problem
<truecoldmind> without this call all is okay
<truecoldmind> timograham, calling cursor in https://github.com/django/django/blob/51890ce8898f821d28f2f6fb6071c936e9bd88f0/django/contrib/gis/db/backends/spatialite/operations.py#L211 causing problems. Does cursor close connection or something else?
...

timograham, when running tests, call to get spatialite version is calling cursor, which creates new connection, which is not a test database connection. I printed connection params, and saw that database name was ":memory:" instead of string which should be for new in-memory databases. It is telling us that test database was not used. You can check it in another way: sqlite allows to not specify database name, and if it will be not specified, and you will try to run `python runtests.py django.contrib.gis.tests.test_geoforms`, it will raise an exception "Please supply the NAME value.".
If I am right, this should be considered as another issue.
(:memory: was printed because it was specified in my settings as NAME)
Or, if running test_geoforms with NAME as some string, and TEST_NAME as :memory:, database file will be created. This is not correct, doesn't it?

[Claude Paroz]

Basically, the problem is the type field of the SpatialiteGeometryColumns model depends on the Spatialite version. And as this happens at import time, the test machinery has not yet done its job.
See ​https://github.com/django/django/blob/51890ce8898f821d28f2f6fb6071c936e9bd88f0/django/contrib/gis/db/backends/spatialite/models.py#L16-L19

It's not the only location in Django where we could take advantage of a lazy model field instanciation, but I'm not aware of any such method currently. Ideas?

[Andriy Sokolovskiy]

An idea how this could be resolved.
​https://gist.github.com/coldmind/be61c330c7e56974bb51

This looks more good than creating meta magic on classes to make these fields to be dynamic

[Tim Graham]

This is a release blocker as it causes the test from #12118 to fail on ​django-master-trusty.

[Andriy Sokolovskiy]

​https://github.com/django/django/pull/3821

[Tim Graham]

Tests don't pass on Jenkins.

[Andriy Sokolovskiy]

Tests are passing now, please review

[Tim Graham <timograham@…>]

In 839f431ef5b927c8b07ce33483b3a7b0dd17b761:

Fixed #24064 -- Prevented database access at compile time in spatialite models.