Post-processing CSS file with Unicode chars fails

[Bouke Haarsma]

Say I have a static file like this: (icons.css)

.icon-home:before { content: '\2302'; } /* '⌂' */
.icon-play-circle2:before { content: '\e048'; } /* '' */
.icon-location:before { content: '\e724'; } /* '' */
.icon-map:before { content: '\e727'; } /* '' */
.icon-twitter-1:before { content: '\f309'; } /* '' */
.icon-facebook-1:before { content: '\f30c'; } /* '' */
.icon-doc:before { content: '📄'; } /* '\1f4c4' */
.icon-calendar:before { content: '📅'; } /* '\1f4c5' */
.icon-search:before { content: '🔍'; } /* '\1f50d' */

When this file is being post-processed (in collectstatic), an exception is raised:

Traceback (most recent call last):
  File "/Applications/PyCharm.app/helpers/pydev/pydevd.py", line 1481, in <module>
    debugger.run(setup['file'], None, None)
  File "/Applications/PyCharm.app/helpers/pydev/pydevd.py", line 1124, in run
    pydev_imports.execfile(file, globals, locals) #execute the script
  File "/Volumes/Users/bouke/Sites/gct/manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "lib/python2.7/site-packages/django/core/management/__init__.py", line 443, in execute_from_command_line
    utility.execute()
  File "lib/python2.7/site-packages/django/core/management/__init__.py", line 382, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "lib/python2.7/site-packages/django/core/management/base.py", line 196, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "lib/python2.7/site-packages/django/core/management/base.py", line 232, in execute
    output = self.handle(*args, **options)
  File "lib/python2.7/site-packages/django/core/management/base.py", line 371, in handle
    return self.handle_noargs(**options)
  File "lib/python2.7/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 163, in handle_noargs
    collected = self.collect()
  File "lib/python2.7/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 120, in collect
    for original_path, processed_path, processed in processor:
  File "lib/python2.7/site-packages/django/contrib/staticfiles/storage.py", line 226, in post_process
    content = pattern.sub(converter, content)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 500: ordinal not in range(128)

[djangsters]

How can I reproduce this. Here it works just fine.

[Łukasz Balcerzak]

Cannot reproduce too. Am closing for now as it doesn't seem like a bug in Django.

Please reopen once you can reproduce. Hints: run collectstatic outside of PyCharm and check css file encoding.

[Eugene]

Reproduce the Error

I encounter the same issues. And after tracing the type,
It happened on ​https://github.com/django/django/blob/1.4/django/contrib/staticfiles/storage.py#L226,
when trying to replace a unicode type into a string type that has a unicode character.

import re

patt = re.compile(r"""(url\(['"]{0,1}\s*(.*?)["']{0,1}\))""")

patt.sub(lambda x: u'The replacement is a function that return unicode', 
    """The content is str with at least a unicode character głó, 
    and don't forget to match url(../some/path)""")

Patching??

I tried to convert the type into string at ​https://github.com/django/django/blob/1.4/django/contrib/staticfiles/storage.py#L172 and it works

return str('url("%s")' % unquote(relative_url))

[Markus Amalthea Magnuson]

Test project for testing collectstatic

[Markus Amalthea Magnuson]

I can't reproduce this. These are the steps I tried:

1. Start a new project.
2. Start a new app.
3. Create a static folder in the new app, and put the icons.css from above in it.
4. Edit settings, setting STATIC_ROOT to e.g. a local folder staticfiles and adding your new app to INSTALLED_APPS`
5. Run collectstatic in your project folder

All files are copied, including icons.css, without errors.

I tested this using the current master branch of Django, on Python 2.7.6, on a Mac.

Attached is my test project, as described.

[anonymous]

Duplicated this error when attempting to 'collectstatic' on:
​http://cdn.jsdelivr.net/semantic-ui/0.18.0/css/semantic.css

Verified that ye0eugene0ey's patch from above resolves the issue.

Python 2.7.3 on Fedora

[Claude Paroz]

Those having succeeded to reproduce, could you please also tell the Django version you are using?

[slumtrimpet <joel@…>]

Good question... it errors on 1.4.1 and works on 1.5.4.
(those are the two env's I have handy)
(I'm 'anonymous' from 6 hours ago)

[Claude Paroz]

Then I guess this is a duplicate of #18430.