Improve `RuntimeError: Max post-process passes exceeded.` error

[Michael]

Having just spend 3 hours trying to debug a collect static issue, to help future travellers, I recommend printing the files that caused max depth to be exceeded, often when a file references itself it causes recursion, but the only clue it currently prints is 'All' which does not narrow down the problem. We can surely do better than that! The proposed change prints the problem files only that keep chaning and can't be resolved:

So instead of getting:

Post-processing 'All' failed!

We get the new and improved:

Post-processing 'jsapp/jsapp/notify.min.js' failed!

Or if more than one file:

Post-processing 'jsapp/jsapp/notify.min.js, jsapp/jsapp/somethingelse.min.js' failed!

I recommend changing from:

# contrib/staticfiles/storage.py line 313: in def post_process(self, paths, dry_run=False, **options):
        unresolved_paths = []
        for i in range(self.max_post_process_passes):
            substitutions = False
            for name, hashed_name, processed, subst in self._post_process(
                paths, adjustable_paths, hashed_files
            ):
                # Overwrite since hashed_name may be newer.
                processed_adjustable_paths[name] = (name, hashed_name, processed)
                if subst and i == self.max_post_process_passes - 1:
                    unresolved_paths.append(name)
                substitutions = substitutions or subst

            if not substitutions:
                break

        if substitutions:
            problem_paths_str = ", ".join(unresolved_paths) if unresolved_paths else "All"
            yield problem_paths_str, None, RuntimeError("Max post-process passes exceeded.")

I recommend changing to (see the comments on the right of the proposed changed lines 1/5 to 5/5):

# contrib/staticfiles/storage.py line 313: in def post_process(self, paths, dry_run=False, **options):
        unresolved_paths = []                                            # < -- add this line 1/5
        for i in range(self.max_post_process_passes):
            substitutions = False
            for name, hashed_name, processed, subst in self._post_process(
                paths, adjustable_paths, hashed_files
            ):
                # Overwrite since hashed_name may be newer.
                processed_adjustable_paths[name] = (name, hashed_name, processed)
                if subst and i == self.max_post_process_passes - 1:      # < -- add this line 2/5
                    unresolved_paths.append(name)                        # < -- add this line 3/5
                substitutions = substitutions or subst

            if not substitutions:
                break

        if substitutions:
            problem_paths_str = ", ".join(unresolved_paths) if unresolved_paths else "All"     # < -- add this line 4/5
            yield problem_paths_str, None, RuntimeError("Max post-process passes exceeded.")   # < -- change this line 5/5

[Sarah Boyce]

Adding the suggestion as a diff

django/contrib/staticfiles/storage.py

@@ -309 +309 @@
 
         paths = {path: paths[path] for path in adjustable_paths}
         substitutions = False
-
+        unresolved_paths = []
         for i in range(self.max_post_process_passes):
             substitutions = False
             for name, hashed_name, processed, subst in self._post_process(
@@ -318 +318 @@
                 # Overwrite since hashed_name may be newer.
                 processed_adjustable_paths[name] = (name, hashed_name, processed)
                 substitutions = substitutions or subst
+                if subst and i == self.max_post_process_passes - 1:
+                    unresolved_paths.append(name)
 
             if not substitutions:
                 break
 
         if substitutions:
-            yield "All", None, RuntimeError("Max post-process passes exceeded.")
+            problem_paths_str = ", ".join(unresolved_paths) if unresolved_paths else "All"
+            yield problem_paths_str, None, RuntimeError("Max post-process passes exceeded.")
 
         # Store the processed paths
         self.hashed_files.update(hashed_files)

Appreciate clear error messages so sounds good to me

[Sarah Boyce <42296566+sarahboyce@…>]

In 2ff0025:

Fixed #35669 -- Improved max post-process passes exceeded error message in HashedFilesMixin.

Signed-off-by: SaJH <wogur981208@…>