get_language_info replaces requested language code with the one found in django.conf.locale.LANG_INFO

[ruffni]

Issue

while following the instructions at
​The set_language redirect view i tried the following:

# settings.py
LANGUAGE_CODE = 'de-ch'                                                                                                                                                                                             
LANGUAGES = [('de-ch', _('Swiss German')),                                                                                                                                                                          
             ('fr-ch', _('Swiss French')),                                                                                                                                                                          
             ('en', _('English'))]
​
# some_template.html
          <form id="set_lang" action="{% url 'set_language' %}" method="post">                                                                                                                                      
            {% csrf_token %}                                                                                                                                                                                        
            <input name="next" type="hidden" />                                                                                                                                                                     
            <li class="nav-item input-group">                                                                                                                                                                       
              <select name="language" class="custom-select" id="lang_selection">                                                                                                                                    
                {% get_current_language as LANGUAGE_CODE %}                                                                                                                                                         
                {% get_available_languages as LANGUAGES %}                                                                                                                                                          
                {% get_language_info_list for LANGUAGES as languages %}                                                                                                                                             
                {% for lang in languages %}                                                                                                                                                                         
                <option value="{{ lang.code }}"                                                                                                                                                                     
                               {% if lang.code == LANGUAGE_CODE %} selected{% endif%}>                                                                                                                              
                  {{ lang.name_local }} ({{lang.code}})</option>                                                                                                                                                    
                {% endfor %}                                                                                                                                                                                        
              </select>                                                                                                                                                                                             
              <div class="input-group-append">                                                                                                                                                                      
                <button class="btn btn-outline-secondary" type="submit" id="set_lang_btn">                                                                                                                          
                  {% translate "Select language" %}</button>                                                                                                                                                        
              </div>                                                                                                                                                                                                
            </li>                                                                                                                                                                                                   
          </form>  

my widget failed to recognize the currently selected language. not sure if i missed something or the documentation doesn't (yet) cover this particular use-case.

Context

the variables set in the template code above are set as follows:

LANGUAGE_CODE = 'fr-ch'

LANGUAGES = [('de-ch', 'allemand suisse'), ('fr-ch', 'franĉais'), ('en', 'anglais')]

languages = [{'bidi': False, 'code': 'de', 'name': 'German', 'name_local': 'Deutsch', 'name_translated': 'Allemand'}, {'bidi': False, 'code': 'fr', 'name': 'French', 'name_local': 'français', 'name_translated': 'Français'}, {'bidi': False, 'code': 'en', 'name': 'English', 'name_local': 'English', 'name_translated': 'anglais'}] 

switching languages and i18n in general work just fine.

Analysis

testing against 'fr-ch' fails because after applying get_language_info_list() to the list of available languages (LANGUAGES) lang.code is the code from the base language found in django.conf.locale.LANG_INFO ('fr' in this case) and not necessarily the code being requested ('fr-ch').

not sure if i'm missing something from the documentation, or if this is intended behavior.

imho getting the information for the base language is fine, but having the option of (easily) checking the currently selected language is necessary.

Test

from django.utils.translation import get_language_info

lang_code = 'fr-ch'
info = get_language_info(lang_code)
assert lang_code == info['code']

Possible Solutions

a) replace the code in the retrieved data to match the requested code

i wasn't able to test this but i guess one could simply add the requested code to the returned LANG_INFO[lang_code] object, i.e. by adding the line:

info['code'] = lang_code[0]

towards the end of django.utils.translation.get_language_info. not sure if this is desired behavior though

b) add documentation on how this particular use-case should be handled

[Claude Paroz]

I think this is expected behaviour, because Django has no fr-ch translations (French translators do not think it's worth maintaining country-specific translations).

If you really want it, you could add any "new" language by following ​https://docs.djangoproject.com/en/stable/topics/i18n/translation/#localization-how-to-create-language-files

I'm closing the ticket, but this does not mean we cannot continue the discussion, if needed.

[ruffni]

aha! i see what i missed to say before:

Replying to Claude Paroz:

If you really want it, you could add any "new" language by following ​https://docs.djangoproject.com/en/stable/topics/i18n/translation/#localization-how-to-create-language-files

this is exactly what i did!

i added the locale directory, made the messages (for language codes 'de-ch' and 'fr-ch' respectively), compiled them (it all works great). but still: comparing LANGUAGE_CODE to get_language_info('de-ch') always fails, see above.
nota bene: i'm not talking about some generic translations or anything alike, i'm talking about translations for languages we provide explicitly

I'm closing the ticket, but this does not mean we cannot continue the discussion, if needed.

i have some doubt about this being intended behavior.

[Claude Paroz]

Would be great to have a sample project to reproduce the issue.

[ruffni]

example project

[ruffni]

Replying to Claude Paroz:

Would be great to have a sample project to reproduce the issue.

added the sample project. it's the same issue with django 3.2 (not sure whether to change the version description of this ticket or not)

[Claude Paroz]

Thanks a lot for the sample project. I'm reopening at least for the missing documentation which should tell that get_language_info and get_language_info_list only return information for languages included in Django.

In your specific use case, I would simply rewrite your language selector as:

<select name="language" class="custom-select" id="lang_selection">
    {% get_current_language as LANGUAGE_CODE %}
    {% get_available_languages as LANGUAGES %}
    {% for code,lang_name in LANGUAGES %}
        <option value="{{ code }}" {% if code == LANGUAGE_CODE %} selected{% endif%}>
            {{ lang_name }} ({{code}})</option>
    {% endfor %}
</select>

If we wanted custom languages data entering Django's LANG_INFO, it would require some deeper refactoring with language data embedded somewhere inside the locale folders (maybe in __init__.py?). Don't know if it's really worth the effort…

[ruffni]

Replying to Claude Paroz:

Thanks a lot for the sample project. I'm reopening at least for the missing documentation which should tell that get_language_info and get_language_info_list only return information for languages included in Django.

yeah, still not completely sure whether this can be sold as intended behavior (to replace a given language code from settings with the code for the language present in the django admin). "languages included in Django" is what irritates imho. this implies languages defined in settings.py are not included?

In your specific use case, I would simply rewrite your language selector as:

<select name="language" class="custom-select" id="lang_selection">
    {% get_current_language as LANGUAGE_CODE %}
    {% get_available_languages as LANGUAGES %}
    {% for code,lang_name in LANGUAGES %}
        <option value="{{ code }}" {% if code == LANGUAGE_CODE %} selected{% endif%}>
            {{ lang_name }} ({{code}})</option>
    {% endfor %}
</select>

thanks for the snippet!
now though, the language selection gets displayed in the currently active language (which is fine in my case where there's only 3 languages). assuming there were a website with dozens of languages with different alphabets and reading directions and whatnot, this would definitely be a deal breaker for usability. imagine trying to find the arabic string for the word "english" in a drop-down menu full of arabic strings -> mission impossible.

If we wanted custom languages data entering Django's LANG_INFO, it would require some deeper refactoring with language data embedded somewhere inside the locale folders (maybe in __init__.py?). Don't know if it's really worth the effort…

i guess some re-working will have to be done to resolve the issue in a nice way.

either defining country-specific translations to be second-class citizens (which django may or may not fall back to if there is no translation for the standard-language; i.e. selecting translations from 'fr-ch' if French is requested but code 'fr' is not found) or completely differentiating between pre-existing languages (the ones you called "included" above) and ignoring them as soon as LANGUAGES is defined in settings.py.