django.setup() should raise an exception instead of hanging on re-entrant calls

[Aymeric Augustin]

#25864 and #26152 discuss issues with calling django.setup() in a file that is imported during the app-loading process. This is actually about apps.populate() -- I wrote django.setup() in the title because it's the public API people are familiar with.

Actually there are three cases.

1. If django.setup() has already completed successfully, calling it again returns immediately.

        if self.ready:
            return

2. If django.setup() is running in another thread, calling it again returns as soon as that other thread has completed the process.

        # populate() might be called by two threads in parallel on servers
        # that create threads before initializing the WSGI callable.
        with self._lock:
            if self.ready:
                return

3. If django.setup() is running in the same thread, calling it must crash. Indeed, django.setup() isn't allowed to return until app-loading has completed. The first call cannot complete without going through the second one, which cannot return, so there's a deadlock. The code tries to detect this condition:

            # app_config should be pristine, otherwise the code below won't
            # guarantee that the order matches the order in INSTALLED_APPS.
            if self.app_configs:
                raise RuntimeError("populate() isn't reentrant")

#26152 suggest that the code hangs instead of raising an exception. Indeed, the second call must block on with self._lock:, since self._lock isn't reentrant. Making it a RLock instead of a Lock should allow the second thread to proceed and to hit that exception.

Also the error condition is incorrect: it won't trigger if the reentrant call happens while importing the first app. (We're reasoning in the single threaded case, which is easy.) Instead populate() should set a private attribute before it starts importing apps and raise the exception if that attribute is set. Alternatively it could check if the RLock is being taken for the second time, but no public API for this appears to exist.

[Aymeric Augustin]

The ticket description is purely based #26152 and on code analysis. I didn't test my theory.

I'd like to investigate further and submit a patch, but I'm not sure when I'll have time for that, so feel free to assign the ticket and work on it if you're interested.

[François Freitag]

PR: ​https://github.com/django/django/pull/7682

[Tim Graham]

Comments for improvement are on the PR.

[Tim Graham <timograham@…>]

In fba4f83:

Fixed #27176 -- Raised an exception for reentrant calls to apps.populate().

Thanks to Aymeric Augustin, Harry Percival, and Tim Graham.