Django UnicodeEncodeError in errorlist

[Andrei <fokau@…>]

I am having an issue with rendering Django's ErrorList if one of my error list items is unicode. When Django renders my errorlist

{{ form.non_field_errors }}

it runs the following code:

class ErrorList(list, StrAndUnicode):
    """
    A collection of errors that knows how to display itself in various formats.
    """
    def __unicode__(self):
        return self.as_ul()

    def as_ul(self):
        if not self: return u''
        return mark_safe(u'<ul class="errorlist">%s</ul>'
                % ''.join([u'<li>%s</li>' % conditional_escape(force_unicode(e)) for e in self])) 

then in force_unicode:

s = unicode(str(s), encoding, errors)

and then translation in lazy:

def __str_cast(self):
    return str(self.__func(*self.__args, **self.__kw))

The problem is that my string contains 'å' symbol and str(u'å') raises UnicodeEncodeError. Is there a good reason why force_unicode and lazy do not use smart_str? I have to do it myself and provide error messages as str objects instead of unicode to make it work.

So I get TemplateSyntaxError Caught UnicodeEncodeError while rendering: 'ascii' codec can't encode character u'\xe5' in position 17: ordinal not in range(128). This seems telling that rendering my error list item (which is u'å') caused the first UnicodeEncodeError having unicode message 'ascii' codec can't encode character u'\xe5' and then second UnicodeEncodeError while rendering the message from the first one. Am I mistaken?

Django version: 1.3.1 (but this seems to happen in 1.4 as well)

Full traceback: ​https://raw.github.com/gist/2499077/ba60cb752acdb429dd6c2814ffb24272037a367a/UnicodeEncodeError.txt

[Claude Paroz]

Unicode in ErrorList is tested here: browser:django/tests/regressiontests/forms/tests/util.py

I'd like to see how/where your error string is defined in your code.
Maybe you can even add the missing test case in the Django test suite, if any?

[Luke Plant]

It sounds like you have some non-ASCII bytestrings, not a unicode object. Note the difference between these:

>>> "é"

which is a not a unicode object, but rather a bytestring equal to "\xc3\xa9" and

>>> u"é"

which is unicode.

To get to that line in force_unicode, you must be passing 'str' not a 'unicode'. Genuine unicode objects will not cause the problem you are describing.