Documentation on providing initial data with migrations is unhelpful

[Rob Atkinson]

​https://docs.djangoproject.com/en/2.0/howto/initial-data/ has a single sentence with links to a RunPython method and low level details, but fails to provide any guidance or links to further documentation necessary to implement loading data.

the suggested requirement is to provide an example of an initial data loading migration, clear description of the implications of naming, and specific instructions for loading additional sample data for testing.

The testing documentation at ​https://docs.djangoproject.com/en/2.0/topics/testing/overview/ also fails to provide useful guidance on how to load test data.

One aspect i was trying to find was whether initial test data for an app would be loaded if i was testing another app that imported and extended the model of that app. This likewise should be covered in the tutorial ​https://docs.djangoproject.com/en/2.0/intro/reusable-apps/ - this should cover how test cases for an app will be used in such a context.

[Tim Graham]

There's nothing particularly special about using migrations to load initial data. Migrations are executed for all test runs as part of creating the test database. I wouldn't use migrations to load test-specific data. For Django's test suite, we use the ORM and TestCase.setUpTestData() for that. Do you want to write up some clarifications based on those suggestions?

[Rob Atkinson]

Happy to suggest some documentation clarifications once I have got my test cases working..

I have been using setUp() for the data, but there is some initial data i want autoloaded in all environments, and as this is part of the app i want to do unit tests on the loading process too. i got a bit lost in the deprecation discussion around fixtures - is this auto loading something we want to do in migrations?

Then i want to add some additional data, currently using setUp() - but setUpTestCase() should be fine - but here the question is the inheritance one - if i am testing an app that extends this app - the perhaps the pattern is to import and extend the test case and inherit these methods too? If the extended add needs to add data - does it override setUpTestCase() and then call super.setUpTestCase()?

Finally (and this is my ultimate use case) - i have an app that allows a user to load a file and process it using metadata they provide. This app is then extended for specialised resource types - so i want a test case for a FileField.

IMHO the testing guidelines should cover (at least with links to more information) the following common patterns

• separation of unit and integration tests (is testing parsing of an uploaded file in an app a unit test? - i can test the parser separately - but i want to unit test the selection and setup of the parser with the uploaded file and other user supplied metadata )

• testing FileField CRUD operations (as these seem to require user code to handle cleanups then testing is vital - ​https://docs.djangoproject.com/en/2.0/topics/files/ does not provide information on how deletions are managed or testing processes - but if its handled automatically and we dont need to test this ourselves then lets add a note to explain that)

• testing views that use external services. (there are various references to mocking and third party tools, but this seems to be a key function and the docs are silent) - for example, we wouldnt have a government application locally that wouldnt use the mandated official external street address validation service

[Tim Graham]

Replying to Rob Atkinson:

I have been using setUp() for the data, but there is some initial data i want autoloaded in all environments, and as this is part of the app i want to do unit tests on the loading process too. i got a bit lost in the deprecation discussion around fixtures - is this auto loading something we want to do in migrations?

That sounds like meets the criteria for "If you want to automatically load initial data for an app, don’t use fixtures. Instead, create a migration for your application with RunPython or RunSQL operations."

Then i want to add some additional data, currently using setUp() - but setUpTestCase() should be fine - but here the question is the inheritance one - if i am testing an app that extends this app - the perhaps the pattern is to import and extend the test case and inherit these methods too? If the extended add needs to add data - does it override setUpTestCase() and then call super.setUpTestCase()?

That sounds fine. This is getting into rather specific issues that I don't think the Django documentation needs to get into.

Finally (and this is my ultimate use case) - i have an app that allows a user to load a file and process it using metadata they provide. This app is then extended for specialised resource types - so i want a test case for a FileField.

IMHO the testing guidelines should cover (at least with links to more information) the following common patterns

• separation of unit and integration tests (is testing parsing of an uploaded file in an app a unit test? - i can test the parser separately - but i want to unit test the selection and setup of the parser with the uploaded file and other user supplied metadata )

This is somewhat beyond the scope of Django's documentation. There might be best practices to follow, but in my opinion, that's left to other resources like books (e.g. Two Scoops of Django - Best Practices for Django X.Y).

• testing FileField CRUD operations (as these seem to require user code to handle cleanups then testing is vital - ​https://docs.djangoproject.com/en/2.0/topics/files/ does not provide information on how deletions are managed or testing processes - but if its handled automatically and we dont need to test this ourselves then lets add a note to explain that)

I don't think so. It might be enhanced in #23251.

• testing views that use external services. (there are various references to mocking and third party tools, but this seems to be a key function and the docs are silent) - for example, we wouldnt have a government application locally that wouldnt use the mandated official external street address validation service

Again, specific use cases like this isn't something that Django has an opinion on.

[Rob Atkinson]

OK - testing style and deeper integration testing with services is out of scope - but i can't see why inheritance of tests isn't a key part of inheritance as a supported mechanism.

There is also a detailed example of using fixtures, yet apparently using fixtures is deprecated "Support for automatic loading of initial_data fixtures and initial SQL data will be removed." all feels a little perverse and unhelpful trying to collate all these disparate pieces of information.

To be consistent, what is needed is an example of how to do a test rather than a simple ref to the low level docs. This should come first, with a reference to the fixtures deprecation issue, and push the example of initial loading of fixtures to a separate page, or even better remove it and reference back to docs for previous versions.

The FileField issue should be explicitly called out with a ref to #23251 and a guideline provided (should custom code for file CRUD handling be unit tested and if so an example of how?)

[Tim Graham]

Regarding inheritance, it's a Python concept rather than a Django-specific one as far as I see.

Fixtures are fine to use, just automatic loading of fixtures named initial_data is removed. You can also use ​TransactionTestCase.fixtures if you want to.

I'm not sure what "example of how to do a test" means exactly.

The discussion has gone in a number of directions and I'm not sure if leaving the ticket open will be useful as the concrete steps to close it are unclear to me. Maybe it would be better to reopen when you have a patch ready.

[Tim Graham]

Please reopen if you have a patch to provide.