in-memory test database does not work with multiprocessing.apply_async

[Tata Krushna Chaitanya]

Hi,
I am using Django 3.0 (On Ubuntu 18.04 installed using pip3). The below example works only with threading but not with multiprocessing.apply_async. Here is my sample, Looking at the Django code I have made sure that all requirements are met, I am using py3.6 and the sqlite3 version is also met.

$ python3 -c "from sqlite3 import dbapi2 as Database; print
(Database.__name__, Database.sqlite_version_info)"

sqlite3.dbapi2 (3, 22, 0)

import os

os.environ["DJANGO_SETTINGS_MODULE"] = "settings"
import datetime, threading
from multiprocessing import Pool

# django  stuff
import django

django.setup()
from polls.models import *
from django.core.mail import mail_admins
from django.test.utils import *
from django.db import connection

def create_object():
    print("Creating Poll")
    p = Question()
    p.question_text = "What's up doc ?"
    p.pub_date = datetime.date.today()
    p.save()
    print("Poll object saved. Id: %d" % p.id)


MODULE = 1

if __name__ == "__main__":
    setup_test_environment()
    old_db_name = "db.sqlite3"
    new_db_name = connection.creation.create_test_db(verbosity=1)
    print("New DATABASE:", new_db_name)
    if MODULE == 0:
        t = threading.Thread(target=create_object)
        t.start()
        t.join()
    elif MODULE == 1:
        with Pool(2) as p:
            p.apply_async(create_object)
    else:
        create_object()
    obj = Question.objects.get()
    print(obj.question_text)
    teardown_test_environment()
    connection.creation.destroy_test_db(old_db_name)

[Tata Krushna Chaitanya]

Similar to https://code.djangoproject.com/ticket/12118 but with multiprocess.apply_async instead of normal threads.

[Simon Charette]

I fear like this won't be possible to do per the in-memory nature of such databases.

It's one thing to serialize queries between multiple threads in a single process but it's another one to coordinate it across multiple processes that need to share the same memory.

Closing as wontfix as nothing seems to indicate this is even possible and Django's core doesn't have any use for it.

[Tata Krushna Chaitanya]

Thanks, Simon. It's a bit obscure use case (Django + multiprocessing) but albiet an important one as the test will fail, currently, I am working around with making the DB updates manually from the tests, we can revisit if there are more users for this.