Accept-Language Header takes precedence over cookie for format localization

[blue-hexagon]

Accept-Language Header takes precedence over language set with cookie

The input localization follows the browsers locale.

In the first image below, my browsers locale is 'da' (Danish/Dansk) while Django language is en-GB - the form inputs should be localized to en-GB and *not* danish!

In the second image, my browsers locale is 'en-GB' and Django language is 'da' (Danish) - the form inputs should be localized to danish and not en-GB!

The images clearly shows the error.

I have provided every relevant piece of information I can think of below the images.

Nowhere in the Django docs, does it mention that form inputs should be localized to the browsers locale.

Image 1

Image 2

Middleware

MIDDLEWARE = [
    "django.middleware.security.SecurityMiddleware",
    "django.middleware.gzip.GZipMiddleware",
    "django.contrib.sessions.middleware.SessionMiddleware",
    'django.middleware.locale.LocaleMiddleware',
    "django.middleware.common.CommonMiddleware",
    "django.middleware.csrf.CsrfViewMiddleware",
    "django.contrib.auth.middleware.AuthenticationMiddleware",
    "django.contrib.messages.middleware.MessageMiddleware",
    "django.middleware.clickjacking.XFrameOptionsMiddleware",
    "django.middleware.common.BrokenLinkEmailsMiddleware",
]

I18N / L10N

TIME_ZONE = "UTC"
LANGUAGE_CODE = "en"
LANGUAGES = [
    ('en', _('English')),
    ('en-gb', _('English')),
    ('da', _('Danish')),
]

USE_I18N = True
LANGUAGE_COOKIE_AGE = 31_536_000  # 1 year.

USE_L10N = True
USE_THOUSAND_SEPARATOR = True
USE_TZ = True

LOCALE_PATHS = [
    BASE_DIR / 'locale/',
]

Loan View

@login_required(login_url="/accounts/login/")
def loan_create(request, institution_id: int):
    if request.POST:
        form = LoanForm(request.POST)
        if form.is_valid():
            form.save()
            messages.add_message(request, messages.SUCCESS, _("Loan created successfully."))
            return redirect(request.META.get("HTTP_REFERER", request.path_info))
        else:
            messages.add_message(request, messages.ERROR, _("An error occured."))
            return redirect(request.META.get("HTTP_REFERER", request.path_info))
    form = LoanFormCreate(
        institution_id=institution_id,
        initial={
            "staff": request.user.id,
            "reminder_has_been_sent": 0,
            "is_returned": False,
        },
    )
    return render(
        request,
        template_name="crud/loan/create.html",
        context={
            "form": form,
        },
    )

Modelforms for Loan

class LoanForm(ModelForm):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)

    class Meta:
        model = Loan
        fields = "__all__"


class LoanFormBase(ModelForm):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)

    class Meta:
        localized_fields = [
            "borrowed_from_date",
            "borrowed_to_date",
            "deposit"
        ]
        model = Loan
        fields = [
            "id",
            "borrower_user",
            "staff_user",
            "item",
            "borrowed_from_date",
            "borrowed_to_date",
            "reminder_has_been_sent",
            "is_returned",
            "extra_information",
            "deposit",
        ]
        widgets = {
            "id": forms.NumberInput(),
            "staff_user": forms.Select(attrs={"class": "form-control"}),
            "borrower_user": forms.Select(attrs={"class": "form-control"}),
            "item": forms.Select(attrs={"class": "form-control", "data-live-search": True}),
            "borrowed_from_date": forms.DateInput(
                attrs={"class": "form-control", "type": "date"}, format=locale_format
            ),
            "borrowed_to_date": forms.DateInput(attrs={"class": "form-control", "type": "date"}),
            "reminder_has_been_sent": forms.NumberInput(attrs={"class": "form-control "}),
            "is_returned": forms.CheckboxInput(attrs={"class": "form-control"}),
            "extra_information": forms.Textarea(attrs={"class": "form-control", "rows": 4}),
            "deposit": forms.TextInput(attrs={"class": "form-control"}),
        }
        labels = {
            "staff_user": _("Lender"),
            "borrower_user": _("Borrower"),
            "item": _("Loaned Asset"),
            "borrowed_from_date": _("Loaned From"),
            "borrowed_to_date": _("Loaned To"),
            "is_returned": _("Returned"),
            "extra_information": _("Comments"),
            "deposit": _("Deposit"),
            "reminder_has_been_sent": _("Reminder Sent"),
        }

        help_texts = {
            "staff_user": _("Select the person responsible for lending the asset to the borrower."),
            "borrower_user": _("Select the person who borrowed the asset."),
            "item": _("Select the asset that has been loaned out."),
            "borrowed_from_date": _("Pick the date when the asset was borrowed."),
            "borrowed_to_date": _("Pick the expected return date for the asset."),
            "is_returned": _("Check this box when the asset has been returned."),
            "extra_information": _("Add any relevant comments or additional information about the loan here."),
            "deposit": _("Enter the deposit amount, if applicable."),
            "reminder_has_been_sent": _("Check if a reminder has been sent for this loan."),
        }


class LoanFormCreate(LoanFormBase):
    def __init__(self, institution_id: int, *args, **kwargs):
        super().__init__(*args, **kwargs)
        # Items shall belong to the institution
        self.fields["item"].queryset = Item.objects.filter(item_group__institution_id=institution_id)

        # Borrower users shall belong to the institution
        self.fields["borrower_user"].queryset = User.objects.filter()
        institution_staff_ids = StaffInstitution.objects.filter().values_list("staff_id")
        self.fields["staff_user"].queryset = User.objects.filter().exclude()

    class Meta(LoanFormBase.Meta):
        localized_fields = [
            "borrowed_from_date",
            "borrowed_to_date",
            "deposit"
        ]
        exclude = ["id", "is_returned"]
        widgets = {
            "extra_information": forms.Textarea(attrs={"class": "form-control", "rows": 4}),
            "borrowed_from_date": forms.DateInput(
                attrs={"class": "form-control", "type": "date"},
            ),
            "borrowed_to_date": forms.DateInput(
                attrs={"class": "form-control", "type": "date"},
            ),
            "item": forms.Select(
                attrs={
                    "class": "form-control selectpicker border",
                    "data-live-search": "true",
                    "data-show-subtext": "true",
                    "data-style": "form-control",
                }
            ),
            "borrower_user": forms.Select(
                attrs={
                    "class": "form-control selectpicker border",
                    "data-live-search": "true",
                    "data-show-subtext": "true",
                    "data-style": "form-control",
                }
            ),
            "staff_user": forms.Select(
                attrs={
                    "class": "form-control selectpicker border ",
                    "data-live-search": "true",
                    "data-show-subtext": "true",
                    "data-style": "form-control",
                    "": "",
                }
            ),
            "reminder_has_been_sent": forms.NumberInput(attrs={"hidden": None}),
        }

Input Dates in Images from HTML file

{# ---------------------------------------------------- #}
{# --- Trying out with and without filters and tags --- #}
{{ form.borrowed_to_date }}
{{ form.borrowed_to_date|localize }}
{% localize off %}
{{ form.borrowed_to_date }}
{% endlocalize %}
{% localize on %}
{{ form.borrowed_to_date }}
{% endlocalize %}
{# ---------------------------------------------------- #}

[Natalia Bidart]

Hello, thank you for your ticket and all the details!

Is there any chance that you could provide a minimal Django project to reproduce this report, including the models? The smallest the example, the better, to be able to isolate the issue and pursue a ticket resolution. Also, could you please try with the latest stable Django (4.2) or the latest pre-release (5.0a1)?

[blue-hexagon]

Replying to Natalia Bidart:

Hello, thank you for your ticket and all the details!

Is there any chance that you could provide a minimal Django project to reproduce this report, including the models? The smallest the example, the better, to be able to isolate the issue and pursue a ticket resolution. Also, could you please try with the latest stable Django (4.2) or the latest pre-release (5.0a1)?

Minimal Django project which reproduces the bug (Django 4.2)

L10n Bug

Accept-Language Header takes precedence over language set with cookie when rendering date input form fields

This is a minimal example of a previously submitted bug - found in the original bug report above.

In the first example, the language and locale is set to en-US but the form fields are localized to da_DK. The docs make no mention that this should be the expected behaviour. I also show that output localizations, such as times and dates strings are localized properly as they should (see green text in images).

Expected formats:

• da_DK uses a dateformat of dd-MM-yyyy
• en_US uses a dateformat of mm/dd/yyyy

Please refer to the Github repository which contains a minimal example as well as the readme, which contains a full analysis with screenshot documentation and relevant highlights of settings, modelforms and models.

​https://github.com/blue-hexagon/dj-L10n_dateinput_bug

[Natalia Bidart]

Thank you blue-hexagon for the test project, I'm cloning it right now to try to reproduce. While I do that, please note that when runninng makemigrations I get this warning:

WARNINGS:
book_store.Book.borrowed_from_date: (fields.W161) Fixed default value provided.
        HINT: It seems you set a fixed date / time / datetime value as default for this field. This may not be what you want. If you want to have the current date as default, use `django.utils.timezone.now`

It seems that the default definition for borrowed_from_date should be the callable datetime.date.today and not the actual result of the call.

[blue-hexagon]

Replying to Natalia Bidart:

Thank you blue-hexagon for the test project, I'm cloning it right now to try to reproduce. While I do that, please note that when runninng makemigrations I get this warning:

WARNINGS:
book_store.Book.borrowed_from_date: (fields.W161) Fixed default value provided.
        HINT: It seems you set a fixed date / time / datetime value as default for this field. This may not be what you want. If you want to have the current date as default, use `django.utils.timezone.now`

It seems that the default definition for borrowed_from_date should be the callable datetime.date.today and not the actual result of the call.

Corrected it to 'timezone.now' and pushed the change.

When testing it, please note the endpoint is: 127.0.0.1:8000/book/create/ and not the root url.

Also note, the new repo is Django 4.2 and not 4.1 as the original ticket is.

[Natalia Bidart]

I have ran and tested the provided Django project. I have reproduced the behavior as described, and after reading the docs, I think this may be a valid issue. Though we would need confirmation from Claude in order to pursue a fix, see below. It's worth noting that there is no issue when the language cookie is not set.

What makes me doubt is that ​the *translation* docs are very clear that the cookie should take precedence over the Accept-Language, for translations, BUT ​the format localization docs say:

Django’s formatting system is capable of displaying dates, times and numbers in templates using the format specified for the current locale. It also handles localized input in forms.
When it’s enabled, two users accessing the same content may see dates, times and numbers formatted in different ways, depending on the formats for their current locale.

But these docs do not mention what is considered their current locale. Is it the cookie? Is it the Accept-Language header?

As a user, my personal expectation is to have the language cookie honoured across a page, including the form date fields. And more so, it feels

[Natalia Bidart]

Tentatively accepting given my rationale above, let's see what Claude thinks :-)

[Claude Paroz]

Is the behavior the same if you remove localized_fields?

[Natalia Bidart]

Replying to Claude Paroz:

Is the behavior the same if you remove localized_fields?

I can confirm the behavior is the same. I have completely commented out the localized_fields definition in the BookForm and the date fields are still showing in the language requested by my browser (es) and not in the language defined by the language cookie (en).

[Claude Paroz]

I had a look at your issue. This is an issue with the <input type="date"> widgets. And this is exactly why Django doesn't use type="date" by default. With those inputs, browsers are forcing their content to the format of the current active browser locale, whatever the language of the page language. So Django cannot do anything to change that.

Read the last comment of #34660 and the tickets/discussion it references.

[Natalia Bidart]

Replying to Claude Paroz:

I had a look at your issue. This is an issue with the <input type="date"> widgets. And this is exactly why Django doesn't use type="date" by default. With those inputs, browsers are forcing their content to the format of the current active browser locale, whatever the language of the page language. So Django cannot do anything to change that.

Read the last comment of #34660 and the tickets/discussion it references.

Thank you Claude, I missed the override for the widget's type attr. TIL many things regarding this topic!

[Michael]

Replying to Claude Paroz:

I had a look at your issue. This is an issue with the <input type="date"> widgets. And this is exactly why Django doesn't use type="date" by default. With those inputs, browsers are forcing their content to the format of the current active browser locale, whatever the language of the page language. So Django cannot do anything to change that.

Read the last comment of #34660 and the tickets/discussion it references.

Correct me if I am understanding you incorrectly, but what I think you are saying is Django does not provide a <input type="date"> because the value should be set according to the active browser's locale, which Django doesn't know server side? If that is what you are saying, I disagree, it is always in the format "YYYY-MM-DD", see:
Date input on MDN: ​https://developer.mozilla.org/en-US/docs/Web/HTML/Element/input/date
Date Time Format on MDN: ​https://developer.mozilla.org/en-US/docs/Web/HTML/Date_and_time_formats#date_strings

HTML uses a variation of the ISO 8601 standard for its date and time strings.

It would be very simple to have Django render date value's server side such that it works correctly with <input type="date">, it just has to do less ....like in Django 4, instead of forcing the default language's DATE_INPUT_FORMAT to the settings language.

I opened an issue about this recently: https://code.djangoproject.com/ticket/35314?replyto=8#comment

[Claude Paroz]

No, if you read the various related tickets and discussions, you'll see that the problem is that browsers are very diverse in the manner they render `<input type="date"> fields. At least it was an issue in the past years (Safari was a typical example). I guess that at some point, we might reconsider this and reevaluate if all browsers render it without usability issues nowadays, and if it's the case we should then implement it in Django.

Please open a discussion on the Django forum if you want to get feedback on this.