runtests.py fails while trying to clean up temp directory on Windows

[Kevin Christopher Henry]

There are two unfortunate aspects to this. One is that most test runs end like this:

Traceback (most recent call last):
  File "runtests.py", line 374, in <module>
    options.failfast, args)
  File "runtests.py", line 220, in django_tests
    teardown(state)
  File "runtests.py", line 188, in teardown
    shutil.rmtree(six.text_type(TEMP_DIR))
  File "C:\Program Other\Python27\Lib\shutil.py", line 252, in rmtree
    onerror(os.remove, fullname, sys.exc_info())
  File "C:\Program Other\Python27\Lib\shutil.py", line 250, in rmtree
    os.remove(fullname)
WindowsError: [Error 32] The process cannot access the file because it is being used by another process:
u'c:\\users\\me\\appdata\\local\\temp\\django_ekeere\\tmprx3hfn'

The other is that the temp directory isn't removed. That's not a severe problem when it comes to temp files and directories, but it's not particularly nice either.

As indicated in the stack trace, the error is caused by the calls to os.remove() in shutil.rmtree() in runtests.teardown(). As indicated in the os.remove() documentation: "On Windows, attempting to remove a file that is in use causes an exception to be raised." This includes files opened by the current process.

The root cause of this, then, is that a number of tests open temporary files but never close them, thus preventing the cleanup process from working properly.

[Kevin Christopher Henry]

I propose a twofold solution:

1. Tracking down the tests that are not closing their open temp files, and making sure they close them.

2. Catching exceptions generated by shutil.rmtree() and printing a helpful message instead of a panicky stack trace.

[Tim Graham]

Could be related to #19905 - "ResourceWarning in formtools tests".

[Kevin Christopher Henry]

I couldn't reproduce the other ticket but I added a note there.

[Kevin Christopher Henry]

See ​https://github.com/django/django/pull/1649.

[Tim Graham <timograham@…>]

In 978e1351a60a0d7b44d367877784109ac01ebc62:

Fixed #21122 -- Improved clean up of test temp directory on Windows

• Fixed test that didn't close the files it opened
• Caught and handled exception when temp directory cannot be removed

[Tim Graham <timograham@…>]

In 02c7dbd2559de4fa09cf6c644a2434e440230c0d:

[1.6.x] Fixed #21122 -- Improved clean up of test temp directory on Windows

• Fixed test that didn't close the files it opened
• Caught and handled exception when temp directory cannot be removed

Backport of 978e1351a6 of master