i18n Javascript code is invalid - lack of Javascript code generator for GAE too

[anonymous]

Function javascript_catalog is invalid why:

1. It is not possible to call this function without use Django compatible request (Python allow to break it but is is nto designed to do it).
2. It is not possible to pass language parameters (need some magic djangoRequest.GETlanguage = self.getLanguage())
3. There IS VERY BAD ASSUMPTION that default language is en - what is false in many regions of globe :) i.e. I have to translate Polish to Polish because loader not load Polish as first language as it defined in settings and django/template uses but loads English for Polish :):):) see INVALID CODE.

Should any default language loaded first.

# first load all english languages files for defaults!!! WHY English is not default deprecate it !!!
for path in paths:

try:

catalog = gettext_module.translation(domain, path, en)
t.update(catalog._catalog)

except IOError:

pass

else:

# 'en' is the selected language and at least one of the packages
# listed in packages has an 'en' catalog
if en_selected:

en_catalog_missing = False

# next load the settings.LANGUAGE_CODE translations if it isn't english
if default_locale != 'en':

for path in paths:

try:

catalog = gettext_module.translation(domain, path, [default_locale])

except IOError:

catalog = None

if catalog is not None:

t.update(catalog._catalog)

# last load the currently selected language, if it isn't identical to the default.
if locale != default_locale:

# If the currently selected language is English but it doesn't have a
# translation catalog (presumably due to being the language translated
# from) then a wrong language catalog might have been loaded in the
# previous step. It needs to be discarded.
if en_selected and en_catalog_missing:

t = {}

else:

locale_t = {}
for path in paths:

try:

catalog = gettext_module.translation(domain, path, [locale])

except IOError:

catalog = None

if catalog is not None:

locale_t.update(catalog._catalog)

if locale_t:

t = locale_t

[anonymous]

Loading default English for JavaScript is invalid by design since Django allow to use different language as default in settings.

[Baptiste Mispelon]

Hi,

I'm having a hard time understanding the issues you're running into.

First, just to make sure, the code you pasted is taken straight from django/views/i18n.py, right? Or did you make modifications in it (I can't find any but the format makes it hard to compare).

Second, regarding the points you raised, I'm going to try and answer them:

1) javascript_catalog is a view so by definition, it takes a request as its first argument.
2) It's related to the point above: this function is meant to be used as a view so that's why it uses a standard way to get the language from the request.
3) I believe the reason that it falls back to English is because since django is written in English, that's the only language that can be guaranteed to have complete translations. Note that English is only used as a default and that it will be overwritten by whatever language you defined in settings.LANGUAGE.

The documentation doesn't mention it but there's a render_javascript_catalog that might be what you're looking for: it's not a view and you can pass it a catalog that you can build yourself (so you can use whatever language you want).

I'm going to mark this as needsinfo.
Please re-open the ticket if you have a concrete example of a failure you encounter with this view, or if I've made errors in my analysis of your problem.

Thanks.

[anonymous]

Thanks for answer.

Some more context: ​http://stackoverflow.com/questions/16099769/how-to-copy-msgid-to-msgstr-in-gettext-to-make-django-i18n-translation

Yes, my code is pasted from django 1.3 include with Google App Engine - it is copy of original django without changes.

ad 1. javascript_catalog is view - sure. I require function generating javascript code not view since using Google App Engine and want use custom handlers and setting langauge methods and think that it will be nice to extract this method/function from javascript_catalog ... I will try render_javascript_catalog since not know it before I do code :)

ad 2. Not understand but it is not match my use case.

ad 3. It is invalid. English is default and CAN NOT BE OVERWRITTEN by ANY LANGUAGE (this bug exist only in JavaScript i18n/not such bug in templates) - i.e. it is impossible to overwite with Polish. Example:

Valid Polish *.po file - NOT WORKS!:

msgid "Polski"
msgstr ""

ARTIFICIALLY valid Polish *.po file - WORKS! - I just copy 'msgid' to 'msgstr':

msgid "Polski"
msgstr "Polski"

IT LOOKS THAT IS REQUIRED TOOL which copy msgid to msgstr if settings.LANGUAGE != 'en' - that is workaround/solution.

ad 3. No reason (fallback not need - not checked but incosistent with template). Templates not not need TRICK whit COPY copy 'msgid' to 'msgstr' but ONLY JAVASCRIPT i18n requires it and "djandojs".

[anonymous]

Not found render_javascript_catalog in django 1.3/1.4.

It looks I need rewrite 'javascript_catalog' code (cut en loading) or just copy 'msgid' to 'msgstr' always.

[anonymous]

Some remark I do nto want use whole django indrastructure - want use templates only and i18n - it is because Google App Engine.

[anonymous]

Finally I make patch breaking false assumption - "English in only one language" for defaults. You could use this code I am giving you license. Consider my previous 4 messages and that you need change some units tests which is done false assumption - it will make i18n js consistent with django templates i18n!

It could patch both 1.3, 1.4 Django not tested others - CODE now is very clear.

NOW ASSUMPTION is that DEFAULT LANGUAGE is DEFAULT LANGUAGE what is true and more intuitive (not Default langauge != always English).

# coding=utf-8
## \authors Cezary K. Wagner

import os
import gettext as gettext_module

from django import http
from django.utils import importlib
from django.conf import settings
from django.utils.translation import check_for_language, activate, to_locale, get_language
from django.utils.text import javascript_quote

from django.views.i18n import LibHead, PluralIdx, SimplePlural, LibFoot,\
  InterPolate, LibFormatHead, get_formats, LibFormatFoot

import logging

def javascript_catalog(request, domain='djangojs', packages=None):
    """
    Returns the selected language catalog as a javascript library.

    Receives the list of packages to check for translations in the
    packages parameter either from an infodict or as a +-delimited
    string from the request. Default is 'django.conf'.

    Additionally you can override the gettext domain for this view,
    but usually you don't want to do that, as JavaScript messages
    go to the djangojs domain. But this might be needed if you
    deliver your JavaScript source from Django templates.
    """
    if request.GET:
        if 'language' in request.GET:
            if check_for_language(request.GET['language']):
                activate(request.GET['language'])
    if packages is None:
        packages = ['django.conf']
    if isinstance(packages, basestring):
        packages = packages.split('+')
    packages = [p for p in packages if p == 'django.conf' or p in settings.INSTALLED_APPS]
    default_locale = to_locale(settings.LANGUAGE_CODE)
    locale = to_locale(get_language())
    t = {}
    paths = []
# FALSE ASSUMPTION START == English is not only one    
#     en_selected = locale.startswith('en')
#     en_catalog_missing = True
# FALSE ASSUMPTION END == English is not only one    
    # paths of requested packages
    for package in packages:
        p = importlib.import_module(package)
        path = os.path.join(os.path.dirname(p.__file__), 'locale')
        paths.append(path)
    # add the filesystem paths listed in the LOCALE_PATHS setting
    paths.extend(list(reversed(settings.LOCALE_PATHS)))
# FALSE ASSUMPTION START == English is not only one
#     # first load all english languages files for defaults
#     for path in paths:
#         try:
#             catalog = gettext_module.translation(domain, path, ['en'])
#             t.update(catalog._catalog)
#         except IOError:
#             pass
#         else:
#             # 'en' is the selected language and at least one of the packages
#             # listed in `packages` has an 'en' catalog
#             if en_selected:
#                 en_catalog_missing = False
#     # next load the settings.LANGUAGE_CODE translations if it isn't english
#     if default_locale != 'en':
#         for path in paths:
#             try:
#                 catalog = gettext_module.translation(domain, path, [default_locale])
#             except IOError:
#                 catalog = None
#             if catalog is not None:
#                 t.update(catalog._catalog)
# FALSE ASSUMPTION END == English is not only one
# TRUE ASSUMPTION START == Default language shoulds be overriden
    for path in paths:
        try:
            catalog = gettext_module.translation(domain, path, [default_locale])
        except IOError:
            catalog = None
        if catalog is not None:
            t.update(catalog._catalog)
# TRUE ASSUMPTION END == Default language shoulds be overriden

    # last load the currently selected language, if it isn't identical to the default.
    if locale != default_locale:
        # If the currently selected language is English but it doesn't have a
        # translation catalog (presumably due to being the language translated
        # from) then a wrong language catalog might have been loaded in the
        # previous step. It needs to be discarded.
# FALSE ASSUMPTION START == English is not only one
#         if en_selected and en_catalog_missing:
#             t = {}
#         else:
#             locale_t = {}
#             for path in paths:
#                 try:
#                     catalog = gettext_module.translation(domain, path, [locale])
#                 except IOError:
#                     catalog = None
#                 if catalog is not None:
#                     locale_t.update(catalog._catalog)
#             if locale_t:
#                 t = locale_t
# FALSE ASSUMPTION END == English is not only one
# TRUE ASSUMPTION START == Default language shoulds be overriden
        locale_t = {}
        for path in paths:
            try:
                catalog = gettext_module.translation(domain, path, [locale])
            except IOError:
                catalog = None
            if catalog is not None:
                locale_t.update(catalog._catalog)
        if locale_t:
            t = locale_t
# TRUE ASSUMPTION END == Default language shoulds be overriden
    src = [LibHead]
    plural = None
    if '' in t:
        for l in t[''].split('\n'):
            if l.startswith('Plural-Forms:'):
                plural = l.split(':',1)[1].strip()
    if plural is not None:
        # this should actually be a compiled function of a typical plural-form:
        # Plural-Forms: nplurals=3; plural=n%10==1 && n%100!=11 ? 0 : n%10>=2 && n%10<=4 && (n%100<10 || n%100>=20) ? 1 : 2;
        plural = [el.strip() for el in plural.split(';') if el.strip().startswith('plural=')][0].split('=',1)[1]
        src.append(PluralIdx % plural)
    else:
        src.append(SimplePlural)
    csrc = []
    pdict = {}
    for k, v in t.items():
        if k == '':
            continue
        if isinstance(k, basestring):
            csrc.append("catalog['%s'] = '%s';\n" % (javascript_quote(k), javascript_quote(v)))
        elif isinstance(k, tuple):
            if k[0] not in pdict:
                pdict[k[0]] = k[1]
            else:
                pdict[k[0]] = max(k[1], pdict[k[0]])
            csrc.append("catalog['%s'][%d] = '%s';\n" % (javascript_quote(k[0]), k[1], javascript_quote(v)))
        else:
            raise TypeError(k)
    csrc.sort()
    for k, v in pdict.items():
        src.append("catalog['%s'] = [%s];\n" % (javascript_quote(k), ','.join(["''"]*(v+1))))
    src.extend(csrc)
    src.append(LibFoot)
    src.append(InterPolate)
    src.append(LibFormatHead)
    src.append(get_formats())
    src.append(LibFormatFoot)
    src = ''.join(src)
    return http.HttpResponse(src, 'text/javascript')

def render_javascript_catalog(language=None, domain='djangojs', packages=None):
    """
    Returns the selected language catalog as a javascript library.

    Receives the list of packages to check for translations in the
    packages parameter either from an infodict or as a +-delimited
    string from the request. Default is 'django.conf'.

    Additionally you can override the gettext domain for this view,
    but usually you don't want to do that, as JavaScript messages
    go to the djangojs domain. But this might be needed if you
    deliver your JavaScript source from Django templates.
    """
#    if request.GET:
#        if 'language' in request.GET:
#            if check_for_language(request.GET['language']):
#                activate(request.GET['language'])
    if language:
      if check_for_language(language):
          activate(language)    
    if packages is None:
        packages = ['django.conf']
    if isinstance(packages, basestring):
        packages = packages.split('+')
    packages = [p for p in packages if p == 'django.conf' or p in settings.INSTALLED_APPS]
    default_locale = to_locale(settings.LANGUAGE_CODE)
    locale = to_locale(get_language())
    t = {}
    paths = []
    en_selected = locale.startswith('en')
    en_catalog_missing = True
    # paths of requested packages
    for package in packages:
        p = importlib.import_module(package)
        path = os.path.join(os.path.dirname(p.__file__), 'locale')
        paths.append(path)
    # add the filesystem paths listed in the LOCALE_PATHS setting
    paths.extend(list(reversed(settings.LOCALE_PATHS)))
#    # first load all english languages files for defaults
#    for path in paths:
#        try:
#            catalog = gettext_module.translation(domain, path, ['en'])
#            t.update(catalog._catalog)
#        except IOError:
#            pass
#        else:
#            # 'en' is the selected language and at least one of the packages
#            # listed in `packages` has an 'en' catalog
#            if en_selected:
#                en_catalog_missing = False
#    # next load the settings.LANGUAGE_CODE translations if it isn't english
#    if default_locale != 'en':
#        for path in paths:
#            try:
#                catalog = gettext_module.translation(domain, path, [default_locale])
#            except IOError:
#                catalog = None
#            if catalog is not None:
#                t.update(catalog._catalog)
    # !do not think if it English or Polish or ... - just load default
    for path in paths:
        try:
            catalog = gettext_module.translation(domain, path, [default_locale])
        except IOError:
            catalog = None
        if catalog is not None:
            t.update(catalog._catalog)
    # last load the currently selected language, if it isn't identical to the default.
    if locale != default_locale:
        # If the currently selected language is English but it doesn't have a
        # translation catalog (presumably due to being the language translated
        # from) then a wrong language catalog might have been loaded in the
        # previous step. It needs to be discarded.
        if en_selected and en_catalog_missing:
            t = {}
        else:
            locale_t = {}
            for path in paths:
                try:
                    catalog = gettext_module.translation(domain, path, [locale])
                except IOError:
                    catalog = None
                if catalog is not None:
                    locale_t.update(catalog._catalog)
            if locale_t:
                t = locale_t
    src = [LibHead]
    plural = None
    if '' in t:
        for l in t[''].split('\n'):
            if l.startswith('Plural-Forms:'):
                plural = l.split(':',1)[1].strip()
    if plural is not None:
        # this should actually be a compiled function of a typical plural-form:
        # Plural-Forms: nplurals=3; plural=n%10==1 && n%100!=11 ? 0 : n%10>=2 && n%10<=4 && (n%100<10 || n%100>=20) ? 1 : 2;
        plural = [el.strip() for el in plural.split(';') if el.strip().startswith('plural=')][0].split('=',1)[1]
        src.append(PluralIdx % plural)
    else:
        src.append(SimplePlural)
    csrc = []
    pdict = {}
    for k, v in t.items():
        if k == '':
            continue
        if isinstance(k, basestring):
            csrc.append("catalog['%s'] = '%s';\n" % (javascript_quote(k), javascript_quote(v)))
        elif isinstance(k, tuple):
            if k[0] not in pdict:
                pdict[k[0]] = k[1]
            else:
                pdict[k[0]] = max(k[1], pdict[k[0]])
            csrc.append("catalog['%s'][%d] = '%s';\n" % (javascript_quote(k[0]), k[1], javascript_quote(v)))
        else:
            raise TypeError(k)
    csrc.sort()
    for k, v in pdict.items():
        src.append("catalog['%s'] = [%s];\n" % (javascript_quote(k), ','.join(["''"]*(v+1))))
    src.extend(csrc)
    src.append(LibFoot)
    src.append(InterPolate)
    src.append(LibFormatHead)
    src.append(get_formats())
    src.append(LibFormatFoot)
    src = ''.join(src)
#    return http.HttpResponse(src, 'text/javascript')
    return src

[Tim Graham]

This looks like a duplicate of #16284.