override_settings for STORAGES loses OPTIONS

[Will Giddens]

If you use override_settings to change STORAGES, any OPTIONS set on the default storage are ignored.

The Settings.__init__() checks if STORAGES is overwritten and sets the DEFAULT_FILE_STORAGE to the new value. StorageHandler.backends checks to see if DEFAULT_FILE_STORAGE is overridden. It proceeds to overwrite the STORAGES["default"] based on the DEFAULT_FILE_STORAGE setting and overrides the explicitly set OPTIONS.

It looks like STATICFILES_STORAGE is handled the same way.

[Will Giddens]

It looks like this is a regression from ​https://github.com/django/django/commit/6b965c600054f970bdf94017ecf2e0e6e0a4326b.

To me, it seems to make more sense to resolve all config elements in Settings instead of StorageHandler to avoid these edge cases.

[Sarah Boyce]

Can you share how you are using override_settings to change STORAGES and the original STORAGES setting? That OPTIONS set on the default storage are ignored when using override_settings to me sounds like expected behaviour

[Will Giddens]

Sure, here's a simplified snippet.

I'm using django-storages to configure S3 storage, but I confirmed Django's built-in storage engines also ignore the OPTIONS. I can provide a test case with the built-in storage engines if it helps.

settings.py:

STORAGES = {
    "default": {
        "BACKEND": "storages.backends.s3.S3Storage",
        "OPTIONS": {
            "bucket_name": "prod-bucket",
            "file_overwrite": False,
        },
    },
    "staticfiles": {
        "BACKEND": "django.contrib.staticfiles.storage.StaticFilesStorage",
    },
}

The test:

@override_settings(
    STORAGES={
        "default": {
            "BACKEND": "storages.backends.s3.S3Storage",
            "OPTIONS": {
                "bucket_name": "test-bucket",
                "file_overwrite": False,
            },
        },
    },
)
class ExampleStorageTestCase(TestCase):
    def test_override_worked(self):
        # This fails with AssertionError: None != 'test-bucket'
        self.assertEqual(default_storage.bucket_name, "test-bucket")

[Sarah Boyce]

I can provide a test case with the built-in storage engines if it helps.

Please do as this is working for me on main:

tests/file_storage/tests.py

@@ -567 +567 @@
                 settings["FILE_UPLOAD_DIRECTORY_PERMISSIONS"],
             )
 
+    @override_settings(
+        STORAGES={
+            DEFAULT_STORAGE_ALIAS: {
+                "BACKEND": "django.core.files.storage.InMemoryStorage",
+                "OPTIONS": {
+                    "location": "explicit_location",
+                    "base_url": "explicit_base_url/",
+                    "file_permissions_mode": 0o666,
+                    "directory_permissions_mode": 0o666,
+                },
+            },
+        }
+    )
+    def test_override_settings(self):
+        self.assertEqual(default_storage._location, "explicit_location")
+        self.assertEqual(default_storage._base_url, "explicit_base_url/")
+        self.assertEqual(default_storage._file_permissions_mode, 0o666)
+        self.assertEqual(default_storage._directory_permissions_mode, 0o666)
+
     def test_file_methods_pathlib_path(self):
         p = Path("test.file")
         self.assertFalse(self.storage.exists(p))

[Will Giddens]

It looks like your test is good- it works on main for me as well. Whenever the DEFAULT_FILE_STORAGE setting was removed this bug goes away.

Since this only affects stable/4.2.x and stable/5.0.x should we pursue a fix or let it be?

[Mariusz Felisiak]

Django 4.2 is in extended support so it no longer receives bugfixes (except security patches), and this issue doesn't qualify for a backport to the Django 5.0. Closing as "invalid".