Error in filebased cache culling

[witek@…]

In line 115 of filebased.py we have:

try:
    filelist = sorted(os.listdir(self._dir))
except (IOError, OSError):
    return

if self._cull_frequency == 0:
    doomed = filelist
else:
    doomed = [os.path.join(self._dir, k) for (i, k) in enumerate(filelist) if i % self._cull_frequency == 0]

In case of _cull_frequency == 0 instead of "doomed = filelist" we should have sth like:

doomed = [os.path.join(self._dir, d) for d in filelist]

Otherwise doomed tuple will not contains absolute paths but relative.

[Jacob]

See also #15825. Not exactly related, but right around in that same part of code and probably worth fixing at the same time.

[Nikolay Zakharov]

Regression test for filebased cache with CULL_FREQUENCY=0

[Nikolay Zakharov]

Initial patch

[Aymeric Augustin]

Could you consolidate the tests and the fix in the same patch?

[Nikolay Zakharov]

Replying to aaugustin:

Could you consolidate the tests and the fix in the same patch?

Yep, sorry for that, just done tests before actually fixing so they came up first.

You can actually pick the merged_patch.diff from #15825, it contains all in one, tests and code for #15806 and #15825.

[Nikolay Zakharov]

United diff (code + tests)

[Kevin Christopher Henry]

This is fixed by the solution to #20536.

[Anssi Kääriäinen <akaariai@…>]

In 7be638390e18fcbfaaed638f9908673360c280d3:

Fixed #20536 -- rewrite of the file based cache backend

• Safer for use in multiprocess environments
• Better random culling
• Cache files use less disk space
• Safer delete behavior

Also fixed #15806, fixed #15825.