[patch] smart_unicode throws UnicodeDecodeError when got a instance with utf-8 encoded string

[nesh <nesh [at] studioquattro [dot] co [dot] yu>]

When smart_unicode get instance instead of string it uses unicode(str(s)) to convert it to string, but if instance returns a utf-8 encoded string you will get a UnicodeDecodeError.

[nesh <nesh [at] studioquattro [dot] co [dot] yu>]

fast fix

[mir@…]

Hi Nesh--thanks for spotting this! I don't understand one thing about your patch, why do you first try to decode from ASCII and only use the DEFAULT_CHARSET when it fails? I'd simply do it like this:

def smart_unicode(s):
    if not isinstance(s, basestring):
        s = unicode(str(s), settings.DEFAULT_CHARSET)
    elif not isinstance(s, unicode):
        s = unicode(s, settings.DEFAULT_CHARSET)
    return s

[nesh <nesh [at] studioquattro [dot] co [dot] yu>]

Yes, that's was my first idea, but I'm not sure why in the first place the str is used so I'm wrapped this in the try...except block for a quick fix.

Also if str call is not essential then we can simply use:

def smart_unicode(s):
   if not isinstance(s, unicode):
      s = unicode(str(s), settings.DEFAULT_CHARSET)
   return s

Regarding the tests, I'll try to add some and send updated patch (with tests) during the day.

[nesh <nesh [at] studioquattro [dot] co [dot] yu>]

smart_unicode patch

[nesh <nesh [at] studioquattro [dot] co [dot] yu>]

test case

[nesh <nesh [at] studioquattro [dot] co [dot] yu>]

Patch is fixed, I also added special case for objects that implement __unicode__ and the test case.

Sorry for double form_utils.diff attachment.

[mir@…]

#3403 marked as duplicate

[Adrian Holovaty]

(In [4522]) Fixed #3314 -- Fixed a bug in newforms smart_unicode. Thanks for the patch, nesh