Translatable URL patterns raise 404 for non-English default language when prefix_default_language=False is used.

[ab]

A simple django project with instruction to replicate the bug can be found here:
​github repo

In brief: prefix_default_language = False raises HTTP 404 for the default unprefixed pages if LANGUAGE_CODE is not "en".

I think the problem is that the function get_language_from_path in django/utils/translation/trans_real.py returns None in case of failure instead of LANGUAGE_CODE: ​diff in 4.2

Consequently, other mechanisms are used to get the language (cookies or headers) that do not work neither.

Related issue with my last comment adding some extra context: https://code.djangoproject.com/ticket/34455

It is the first time I contribute to django, I hope the bug report is OK. I am also willing to write the patch and test if required.

[ab]

Expected behavior: ​django 4.2 documentation

LocaleMiddleware tries to determine the user’s language preference by following this algorithm:

• First, it looks for the language prefix in the requested URL. This is only performed when you are using the i18n_patterns function in your root URLconf. See Internationalization: in URL patterns for more information about the language prefix and how to internationalize URL patterns.

• Failing that, it looks for a cookie. The name of the cookie used is set by the LANGUAGE_COOKIE_NAME setting. (The default name is django_language.)

• Failing that, it looks at the Accept-Language HTTP header. This header is sent by your browser and tells the server which language(s) you prefer, in order by priority. Django tries each language in the header until it finds one with available translations.

• Failing that, it uses the global LANGUAGE_CODE setting.

[Mariusz Felisiak]

Thanks for the report. The use of URL patterns marked as translatable is crucial for this bug.

Regression in 94e7f471c4edef845a4fe5e3160132997b4cca81.
Reproduced at c24cd6575f948661fa0ed8b27b79098610dc3ccc.

[Mariusz Felisiak]

Replying to ab:

Expected behavior: ​django 4.2 documentation

LocaleMiddleware tries to determine the user’s language preference by following this algorithm:

• First, it looks for the language prefix in the requested URL. This is only performed when you are using the i18n_patterns function in your root URLconf. See Internationalization: in URL patterns for more information about the language prefix and how to internationalize URL patterns.

• Failing that, it looks for a cookie. The name of the cookie used is set by the LANGUAGE_COOKIE_NAME setting. (The default name is django_language.)

• Failing that, it looks at the Accept-Language HTTP header. This header is sent by your browser and tells the server which language(s) you prefer, in order by priority. Django tries each language in the header until it finds one with available translations.

• Failing that, it uses the global LANGUAGE_CODE setting.

IMO it still works that way. However, in Django 4.2 get_language_from_request() returns the language from a request (en for me) which is activated and the default path about/ is no longer translated to the a-propos/. This is definitely a change from the previous behavior.

[ab]

Replying to Mariusz Felisiak:

IMO it still works that way. However, in Django 4.2 get_language_from_request() returns the language from a request (en for me) which is activated and the default path about/ is no longer translated to the a-propos/. This is definitely a change from the previous behavior.

Thank you Mariusz for the quick reaction.

I agree it still globally works that way, nevertheless, in the case I describe, when django looks for the language prefix in the requested URL and there is not language prefix, I would expect django to return "fr", not to go to the next steps of the algorithm. Because I want prefix_default_language = False to take precedence on cookies or headers. Does it make sense?

I need to add that I use translate_url to ​build the links in my templates.
Consequently, my URLs are translated in the template only (hence the 404).

So you're right about the default path not being translated anymore.

[Sarah Boyce]

I have a PR with what I think the issue is, but not confident ​https://github.com/django/django/pull/16797
@ab I think what you're saying makes sense

[David Sanders]

I agree it still globally works that way, nevertheless, in the case I describe, when django looks for the language prefix in the requested URL and there is not language prefix, I would expect django to return "fr", not to go to the next steps of the algorithm. Because I want prefix_default_language = False to take precedence on cookies or headers. Does it make sense?

My 2¢: Is ignoring the process for determining the language the least surprising choice here though? It all depends on whether no-prefix URL should refer to a user setting or the site's default language. I mean imho navigating to a prefix-less URL I might expect it to show the language I chose 🤷‍♂️

[ab]

@Sarah: yes, it is the same problem. After investigating the code, the change in behavior is linked to the fact that get_language_from_path returns None when the url is not prefixed. So, the cookie is used or the Accept-Language header sent by the browser. In my case, I think it is the HTTP header.

@David: thanks for your contribution, but I do not fully agree. If prefix_default_url is True, the language is correctly detected by django based on the URL. If I set prefix_default_url to False I expect the same behavior for the default language without prefix. When I decide do use i18n_patterns at least (I have just added this tag to the ticket).
When i18n_patternsis not used, I agree with you.

So the problem might come from i18n_patterns not calling/handling correctly the calls to the new get_language_* functions.

[ab]

@sarah: I'll test your patch because your edits might solve the problem with HTTP headers too. Thanks!

[ab]

Just to keep track of the current work on this issue, there is a discussion about how django should behave here: ​https://github.com/django/django/pull/16797#issuecomment-1524958085

As suggested by Sarah, I'll post to django-developers for a wider range of opinions.

[Mariusz Felisiak]

​Django forum discussion.

[Mariusz Felisiak]

Alternative ​PR.

[Mariusz Felisiak <felisiak.mariusz@…>]

In 0e444e8:

Fixed #34515 -- Made LocaleMiddleware prefer language from paths when i18n patterns are used.

Regression in 94e7f471c4edef845a4fe5e3160132997b4cca81.

This reverts commit 94e7f471c4edef845a4fe5e3160132997b4cca81
(refs #34069) and
partly reverts commit 3b4728310a7a64f8fcc548163b0aa5f98a5c78f5.

Thanks Anthony Baillard for the report.

Co-Authored-By: Sarah Boyce <42296566+sarahboyce@…>

[Mariusz Felisiak <felisiak.mariusz@…>]

In f200d836:

[4.2.x] Fixed #34515 -- Made LocaleMiddleware prefer language from paths when i18n patterns are used.

Regression in 94e7f471c4edef845a4fe5e3160132997b4cca81.

This reverts commit 94e7f471c4edef845a4fe5e3160132997b4cca81
(refs #34069) and
partly reverts commit 3b4728310a7a64f8fcc548163b0aa5f98a5c78f5.

Thanks Anthony Baillard for the report.

Co-Authored-By: Sarah Boyce <42296566+sarahboyce@…>

Backport of 0e444e84f87d174713a2aef0c4f9704ce2865586 from main

[ab]

Thank you for working on this and maintaining django so well!

Anthony