Opened 7 years ago

Last modified 7 years ago

#28153 closed New feature

patch: it's now easier to populate data databases in test databases that run code in parallel — at Version 1

Reported by: Marcos Diez Owned by: nobody
Component: Testing framework Version: dev
Severity: Normal Keywords:
Cc: Triage Stage: Unreviewed
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description (last modified by Marcos Diez)

Although Django makes very easy for one to extend django.test.runner.DiscoverRunner , it's setup_databases() does too much.

Currently, it

  • creates all the test databases (for single thread unit tests)
  • duplicates all the test databases (in case of parallel unit tests)

In case I am running not running tests in parallel, I can just populate the DB after running unit tests without any issues.

But if I care about my time and want to run tests in parallel, I can either:

a) populate my data after setup_databases() is executed, once for each thread of the parallel tests, which is slow
b) get my hands dirty and reimplement setup_databases()

I propose (and I am sending the code to do so) a better solution. We just have to break setup_databases() in 3 functions:

DiscoverRunner.prepare_databases()
DiscoverRunner.populate_databases() # noop by default
DiscoverRunner.duplicate_databases_if_necessary()

The idea is quite simple: in order to be backward compatible, setup_databases() , will still exist but only call three functions above in that order.

The first function will create all the test databases necessary for non parallel tests to run.

populate_databases() , which should be a no op, can be overwritten by the user who extends django.test.runner.DiscoverRunner so his/her data can be populated

Afterwards, all the test DBs are copied as many times as necessary in case parallel tests are run via DiscoverRunner.duplicate_databases_if_necessary()

I believe this change on Django will have no downside, will be backward compatible and help people who needs to populate real data on the DB for their tests.

The PR with the changes for this to work is available here: https://github.com/django/django/pull/8437

Change History (1)

comment:1 by Marcos Diez, 7 years ago

Description: modified (diff)
Note: See TracTickets for help on using tickets.
Back to Top