Tests fail when i18n set to True.

[wassup]

Hi,

When i18n is set to True and project language is not English, some of the default django tests do not pass (it worked fine in 1.3). I believe the reason is that the messages get translated and localized before the tests are run, which results in e.g.:

======================================================================
FAIL: test_unusable_password (django.contrib.auth.tests.forms.PasswordResetFormTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/django/contrib/auth/tests/forms.py", line 336, in test_unusable_password
    [u"The user account associated with this e-mail address cannot reset the password."])
AssertionError: [u'U\u017cytkownik, kt\xf3rego konto powi\u0105zane jest z tym adresem e-mail nie mo\u017ce zresetowa\u0107 has\u0142a.'] != [u'The user account associated with this e-mail address cannot reset the password.']

======================================================================
FAIL: test_confirm_different_passwords (django.contrib.auth.tests.views.PasswordResetTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/django/contrib/auth/tests/views.py", line 184, in test_confirm_different_passwords
    self.assertContainsEscaped(response, SetPasswordForm.error_messages['password_mismatch'])
  File "/usr/lib/python2.7/site-packages/django/contrib/auth/tests/views.py", line 54, in assertContainsEscaped
    return self.assertContains(response, escape(force_unicode(text)), **kwargs)
  File "/usr/lib/python2.7/site-packages/django/test/testcases.py", line 637, in assertContains
    msg_prefix + "Couldn't find '%s' in response" % text)
AssertionError: Couldn't find 'Hasła się nie zgadzają.' in response

======================================================================
FAIL: test_email_not_found (django.contrib.auth.tests.views.PasswordResetTest)
Error is raised if the provided email address isn't currently registered
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/django/contrib/auth/tests/views.py", line 91, in test_email_not_found
    self.assertContainsEscaped(response, PasswordResetForm.error_messages['unknown'])
  File "/usr/lib/python2.7/site-packages/django/contrib/auth/tests/views.py", line 54, in assertContainsEscaped
    return self.assertContains(response, escape(force_unicode(text)), **kwargs)
  File "/usr/lib/python2.7/site-packages/django/test/testcases.py", line 637, in assertContains
    msg_prefix + "Couldn't find '%s' in response" % text)
AssertionError: Couldn't find 'Ten adres e-mail nie ma przypisanego konta. Jesteś pewien, że zarejestrowałeś się?' in response

----------------------------------------------------------------------

I believe the default django tests should not be run against the localized strings or am I missing something?

Regards,

wassup

[Claude Paroz]

I can confirm the bug in test_unusable_password (the tested string should be translated itself).

However I'm unable to reproduce the PasswordResetTest errors, as the AuthViewsTestCase parent test case is setting 'en' as default language. Could you debug test_confirm_different_passwords and check if SetPasswordForm.error_messages['password_mismatch'] is a proxy string before the assert statement?

[Jannis Leidel]

Fixed formatting, please use Trac's preview feature.

[wassup]

@jezdez: Sorry - tried to edit it after posting, but it was not possible. My fault.

@claudep: SetPasswordForm.error_messages['password_mismatch'] before assert is a proxy object: <django.utils.functional.proxy object at 0x11b2310>. When I treat it with a loop to get the content, the string is already translated to: 'Hasła się nie zgadzają.'

[Claude Paroz]

OK, now the question is which language is active when the proxy string is translated to the real string. It should be 'en' and apparently it is not. Could you also debug the current language in the test (from django.utils.translation import get_language; print get_language())?

If it is not 'en', it may be that you are modifying the current language somewhere in your code, either in a view or in a middleware.

[wassup]

It spits out 'en'. Additionally, I thought it shall be fixed in one place, but apparently we're progressing on the bug by bug way. Thus, there are a couple of more to go that also fail on an already translated string:

======================================================================
FAIL: test_password_change_fails_with_invalid_old_password (django.contrib.auth.tests.views.ChangePasswordTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/django/contrib/auth/tests/views.py", line 209, in test_password_change_fails_with_invalid_old_password
    self.assertContainsEscaped(response, PasswordChangeForm.error_messages['password_incorrect'])
  File "/usr/lib/python2.7/site-packages/django/contrib/auth/tests/views.py", line 54, in assertContainsEscaped
    return self.assertContains(response, escape(force_unicode(text)), **kwargs)
  File "/usr/lib/python2.7/site-packages/django/test/testcases.py", line 637, in assertContains
    msg_prefix + "Couldn't find '%s' in response" % text)
AssertionError: Couldn't find 'Podane stare hasło jest niepoprawne. Proszę podać je jeszcze raz.' in response

======================================================================
FAIL: test_password_change_fails_with_mismatched_passwords (django.contrib.auth.tests.views.ChangePasswordTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/django/contrib/auth/tests/views.py", line 219, in test_password_change_fails_with_mismatched_passwords
    self.assertContainsEscaped(response, SetPasswordForm.error_messages['password_mismatch'])
  File "/usr/lib/python2.7/site-packages/django/contrib/auth/tests/views.py", line 54, in assertContainsEscaped
    return self.assertContains(response, escape(force_unicode(text)), **kwargs)
  File "/usr/lib/python2.7/site-packages/django/test/testcases.py", line 637, in assertContains
    msg_prefix + "Couldn't find '%s' in response" % text)
AssertionError: Couldn't find 'Hasła się nie zgadzają.' in response

======================================================================
FAIL: test_password_change_succeeds (django.contrib.auth.tests.views.ChangePasswordTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/django/contrib/auth/tests/views.py", line 230, in test_password_change_succeeds
    self.fail_login()
  File "/usr/lib/python2.7/site-packages/django/contrib/auth/tests/views.py", line 196, in fail_login
    self.assertContainsEscaped(response, AuthenticationForm.error_messages['invalid_login'])
  File "/usr/lib/python2.7/site-packages/django/contrib/auth/tests/views.py", line 54, in assertContainsEscaped
    return self.assertContains(response, escape(force_unicode(text)), **kwargs)
  File "/usr/lib/python2.7/site-packages/django/test/testcases.py", line 637, in assertContains
    msg_prefix + "Couldn't find '%s' in response" % text)
AssertionError: Couldn't find 'Proszę wpisać poprawną nazwę użytkownika i hasło. Uwaga: wielkość liter ma znaczenie.' in response

[Claude Paroz]

In [17811]:

Make auth test pass even when LANGUAGE_CODE is not 'en'. Refs #17980. Thanks wassup for the report.

[Claude Paroz]

I fixed the issue about test_unusable_password. Now you should really find out why you obtain localized strings in assertContainsEscaped in your tests. I cannot reproduce the problem.

[wassup]

It seems that despite the fact that get_language() returns 'en', the string is already translated into Polish in this case. Any hints what it might be caused by? I use the stable of version of django 1.4, not the one from svn. Shall I possibly try the development version and report back?

[Claude Paroz]

I still cannot figure out how those strings get translated before the tests. Do you manipulate these error_message dicts elsewhere in your code? And no, I doubt that anything in the development version would fix this.

[wassup]

I do not touch the error_message dicts anywhere in my code. Exactly the same version of my project passed the tests well in django 1.3. I just have set my project language to 'pl' and operate on ordinary {%trans 'string'%} and _('translate this as well') ('_' being gettext_lazy or ugettext).

[wassup]

I believe I have found the reason why it is not working for me - the problem was in django.contrib.auth.forms. Hence, I provide a tiny patch that solved the issue in my case.

[Claude Paroz]

Sorry, but no, the patch is not correct. Using ugettext_lazy is purposely allowing some strings to be translated as late as possible to honour the current language.

Well, would you mind sending your application code to me personally (claude at 2xlibre dot net), so as I can try to reproduce and debug with it?

[wassup]

@claudep: Let me just get the consent of the other members of the team. However, I've found a similar bug #17562. The same happens here, inter alia, with fail_login().

[wassup]

One more thing, don't really know whether it would be relevant. If ugettext_lazy is used, it returns __proxy__ object. When I loop through it, the error message is already translated. Ugettext returns a string, the error message is translated as well. Yet, the first one fails on assert, while the second one does not. I'm still waiting for the response of the other project members and as soon as I get them, I will send you the sources.

[Claude Paroz]

Interesting case :-) At least, I've now been able to reproduce the issue. It depends on other tests doing string translations for a language other than 'en' before the auth tests, that's why I couldn't reproduce it.

So let's assume tests are run before auth tests, and they activate a translation ('pl' in our case). Now the cached django.utils.translation.trans_real._default global variable is set to Polish. Just changing LANGUAGE_CODE in setUp is evidently not sufficient to change anything. So let's assume we fix this by calling activate('en') after setting LANGUAGE_CODE to 'en' (still in testcase setUp). Unfortunately, this is not sufficient, because after self.client.login(), the translation will be deactivated again (due to a deactivate() call in LocaleMiddleware.process_response). So when the __proxy__ string get translated in assertContainsEscaped, the translation engine try the _active translation first (now unset), then the _default cached translation objects and BANG... the Polish translation is used.

Now what about solutions?
I think that the proper way to fix the issue would be to use the setting_changed signal to reset the _default cached variable when LANGUAGE_CODE is changed, and use override_settings for AuthViewsTestCase.
However, this might seem too invasive to enter a stable release, so I suggest to apply a more simple patch, less clean, but functional that could be applied for 1.4.1. And then the more elaborate patch should be committed for 1.5. Patches follow.

[Claude Paroz]

Simple patch: reset _default global variable

[Claude Paroz]

Use a signal to properly handle LANGUAGE_CODE change

[Anssi Kääriäinen]

FWIW the signal patch doesn't look _that_ invasive to me, so I am +0 on backpatching it.

[Claude Paroz]

In [17819]:

[1.4.X] Make auth test pass even when LANGUAGE_CODE is not 'en'. Refs #17980. Thanks wassup for the report.

Backport of r17811 from trunk.

[Claude Paroz]

#18395 has a related issue.

[Claude Paroz]

I think this should be fixed now that #18395 has been fixed.