OverwritingStorageTests fail if a TemporaryUploadedFile is used

[bcail]

Ticket #28144 added the option of using custom flags on a storage object to allow overwriting files in storage. However, this doesn't seem to work for temporary uploaded files, since the ​alternate path is taken in the _save method.

Here is an example test that fails for me - it loops forever:

diff --git a/tests/file_storage/tests.py b/tests/file_storage/tests.py
index 420314573d..d404300708 100644
--- a/tests/file_storage/tests.py
+++ b/tests/file_storage/tests.py
@@ -648,6 +648,34 @@ class OverwritingStorageTests(FileStorageTests):
         finally:
             self.storage.delete(name)
 
+    def test_save_overwrite_behavior_temp_file(self):
+        """Saving to same file name twice overwrites the first file."""
+        name = "test.file"
+        self.assertFalse(self.storage.exists(name))
+        content_1 = b"content one"
+        content_2 = b"second content"
+        f_1 = TemporaryUploadedFile('tmp1', 'text/plain', 11, 'utf8')
+        f_1.write(content_1)
+        f_1.seek(0)
+        f_2 = TemporaryUploadedFile('tmp2', 'text/plain', 14, 'utf8')
+        f_2.write(content_2)
+        f_2.seek(0)
+        stored_name_1 = self.storage.save(name, f_1)
+        try:
+            self.assertEqual(stored_name_1, name)
+            self.assertTrue(self.storage.exists(name))
+            self.assertTrue(os.path.exists(os.path.join(self.temp_dir, name)))
+            with self.storage.open(name) as fp:
+                self.assertEqual(fp.read(), content_1)
+            stored_name_2 = self.storage.save(name, f_2)
+            self.assertEqual(stored_name_2, name)
+            self.assertTrue(self.storage.exists(name))
+            self.assertTrue(os.path.exists(os.path.join(self.temp_dir, name)))
+            with self.storage.open(name) as fp:
+                self.assertEqual(fp.read(), content_2)
+        finally:
+            self.storage.delete(name)

[alexialg05]

can i be assigned on this issue?

[bcail]

@alexialg05, yes, you can assign yourself to this issue, but we should probably see if others confirm this as an issue. If it is confirmed as an issue, there may be design questions to figure out.

[Natalia Bidart]

Hello bcail, thank you for your report!

This definitely seems like a valid issue, file_move_safe (from django.core.files.move) requires the allow_overwrite flag to be passed to perform overwrites. I don't have clarity on the possible fix, I agree that it requires a design decision. A forum post seeking advice would be the best next step, I think.

[bcail]

Thanks, @Natalia. I opened a new ​forum post.

[bcail]

I opened a ​draft PR with a possible direction we could go.

[Sarah Boyce <42296566+sarahboyce@…>]

In 0b33a3a:

Fixed #35326 -- Added allow_overwrite parameter to FileSystemStorage.

[Sarah Boyce <42296566+sarahboyce@…>]

In 480ccf90:

Refs #35326 -- Made cosmetic edits to 5.1 release notes.

[Sarah Boyce <42296566+sarahboyce@…>]

In 8d6a20b:

Fixed #35604, Refs #35326 -- Made FileSystemStorage.exists() behaviour independent from allow_overwrite.

Partially reverts 0b33a3abc2ca7d68a24f6d0772bc2b9fa603744e.

Storage.exists(name) was documented to "return False if
the name is available for a new file." but return True if
the file exists. This is ambiguous in the overwrite file
case. It will now always return whether the file exists.

Thank you to Natalia Bidart and Josh Schneier for the
review.

[Sarah Boyce <42296566+sarahboyce@…>]

In e42defb:

[5.1.x] Fixed #35604, Refs #35326 -- Made FileSystemStorage.exists() behaviour independent from allow_overwrite.

Partially reverts 0b33a3abc2ca7d68a24f6d0772bc2b9fa603744e.

Storage.exists(name) was documented to "return False if
the name is available for a new file." but return True if
the file exists. This is ambiguous in the overwrite file
case. It will now always return whether the file exists.

Thank you to Natalia Bidart and Josh Schneier for the
review.

Backport of 8d6a20b656ff3fa18e36954668a44a831c2f6ddd from main.

[nessita <124304+nessita@…>]

In 47f18a72:

Refs #35326 -- Adjusted deprecation warning stacklevel in FileSystemStorage.OS_OPEN_FLAGS.

[Natalia <124304+nessita@…>]

In 8f5d2c37:

[5.1.x] Refs #35326 -- Adjusted deprecation warning stacklevel in FileSystemStorage.OS_OPEN_FLAGS.

Backport of 47f18a722624527cc72eef44cfc9d1e07ea4b4e0 from main.