Separate test-collection setup from runtests.py's setup() for use in get_app_test_labels()

[Chris Jerdonek]

This is another clean-up follow-up to ​PR #4106, similar to ​PR #14276.

TLDR: Refactor out from runtests.py's 125-line ​setup() and ​teardown() simpler setup_test_collection() and teardown_test_collection() functions for use in the new ​get_app_test_labels() (used for bisect_tests() and paired_tests()). (The exact names aren't so important.)

Longer version:

Currently, runtests.py's setup() function and its role in getting the list of default test labels for bisect_tests(), paired_tests(), and ​test_runner.run_tests() is a bit complicated and harder to understand than it needs to be.

There are a couple reasons for this. First, setup() is actually doing two kinds of setup: one for collecting the default test labels, and one for setting up the test run (needed only for ​django_tests() but never bisect_tests() and paired_tests()). Second, the way the list of default test labels is obtained is a bit roundabout. Namely, a bunch of apps are added to INSTALLED_APPS, and then runtests.py's get_installed() is called to read from INSTALLED_APPS. However, for test-collection, INSTALLED_APPS doesn't actually need to be modified. You can see a side effect of this in the fact that get_installed() doesn't just return test labels. It also returns the following modules, which no longer contain any tests (this is the thing that ​PR #4106 from six years ago fixed):
django.contrib.admin, django.contrib.auth, django.contrib.contenttypes, django.contrib.flatpages, django.contrib.messages, django.contrib.redirects, django.contrib.sessions, django.contrib.sites, django.contrib.staticfiles.

By extracting out from setup() a setup_test_collection() function that just collects and returns the top-level, filtered test labels, I think things will become a lot easier to understand and work with -- not just for bisect_tests() and paired_tests(), but also for the main django_tests() case.

[Carlton Gibson]

Yes, thank you for the good explanation Chris, this makes sense.

By extracting out from setup() a setup_test_collection() function that just collects and returns the top-level, filtered test labels, I think things will become a lot easier to understand and work with -- not just for bisect_tests() and paired_tests(), but also for the main django_tests() case.

I agree setup() is a little opaque so +1.
Thanks.

[Chris Jerdonek]

Here is a first PR: ​https://github.com/django/django/pull/14477

[Mariusz Felisiak <felisiak.mariusz@…>]

In 90f41c2:

Refs #32668 -- Made setup()'s test_labels argument optional in runtests.py.

[Mariusz Felisiak <felisiak.mariusz@…>]

In ffc0d57a:

Refs #32668 -- Refactored away module_found_in_labels in runtests.py's setup().

[Mariusz Felisiak <felisiak.mariusz@…>]

In 0d281613:

Refs #32668 -- Simplified get_test_modules() in runtests.py.

This simplifies runtests.py's get_test_modules() in a few ways. For
example, it changes the function to yield strings instead of returning
pairs of strings, which simplifies the calling code.

This commit also changes SUBDIRS_TO_SKIP from a list to a dict since
the directories to skip depend on the parent directory.

[Chris Jerdonek]

PR: ​https://github.com/django/django/pull/14484

[GitHub <noreply@…>]

In 9812b486:

Refs #32668 -- Simplified start_at/start_after logic in runtests.py's setup().

[Mariusz Felisiak <felisiak.mariusz@…>]

In 9b9cea04:

Refs #32668 -- Added gis_enabled argument to get_test_modules().

[Mariusz Felisiak <felisiak.mariusz@…>]

In b3083d5b:

Refs #32668 -- Refactored out setup_collect_tests() in runtests.py.

[Mariusz Felisiak <felisiak.mariusz@…>]

In e96e936:

Refs #32668 -- Passed setup()'s return value to run_tests() instead of get_installed().

[Mariusz Felisiak <felisiak.mariusz@…>]

In 9389d4d3:

Refs #32668 -- Changed bisect_tests() and paired_tests() to use only setup_collect_tests().

[Mariusz Felisiak <felisiak.mariusz@…>]

In 87bb746e:

Refs #32668 -- Renamed setup()/teardown() to setup_run_tests()/teardown_run_tests() in runtests.py.