CachedFilesMixin crashed when CSS files contains non-ascii utf-8 character

[brantyoung]

Traceback (most recent call last):

...
...
  File ".../ENV/local/lib/python2.7/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 120, in collect
    for original_path, processed_path, processed in processor:
  File ".../ENV/local/lib/python2.7/site-packages/django/contrib/staticfiles/storage.py", line 226, in post_process
    content = pattern.sub(converter, content)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xf0 in position 366: ordinal not in range(128)

Reproduce：

# save a css file that contains non-ascii utf-8 character (ie Chinese character):

.icon-link:before { content: "错"; }

# set STATICFILES_STORAGE to any storage class that will involve CachedFilesMixin mixin
# run manage.py collectstatic

Howto fix:

I relace following code:

content = original_file.read().decode

to:

content = original_file.read().decode('utf-8')

just worked as expected.

[Danilo Bargen]

I can't reproduce this on current trunk using STATICFILES_STORAGE = 'django.contrib.staticfiles.storage.CachedStaticFilesStorage' (which uses the CachedFilesMixin). I even tried converting the css file to latin1, which garbled the contents, but still worked using collectstatic.

Could you provide a CSS file that causes problems for you?

Or can somebody else reproduce the problem?

[Aymeric Augustin]

Closing as 'needsinfo' since there isn't enough information to reproduce the issue.

[brantyoung]

I create a minimal project to reproduce this ticket: ​https://bitbucket.org/brantyoung/cached_files_mixin_bug/overview

It's weird, if I moving all of font files to any directory except *PROJECTROOT*/css, every thing works fine.

But if both iconic-font.css and font files in the same sub-directory, CachedFilesMixin will crash.

[Jannis Leidel <jannis@…>]

In [3727f6d09681e4cb23d67e14ecc677a364c991bd]:

Fixed #18430 -- Use the FILE_CHARSET setting when reading from a file during post processing with the cached staticfiles storage. Thanks to Brant Young for initial debugging.