I18N needs improvements

[Marc Garcia]

As described in ​http://vaig.be/2008/07/26/django-i18n-status/ Django's i18n has many features missing.

This ticket is a proposal to:

• Customize first day of week in admin's calendar (based on current locale).
• Customize date/datetime formats when a date/datetime is displayed (based on current locale).
• Customize number format when a decimal number is displayed (based on current locale).
• Customize input format for date/datetime widgets (based on current locale). It also includes displaying existing data in the same format, when shown on the widget.
• Customize input format for decimal number widgets (based on current locale). It also includes displaying existing data in the same format, when shown on the widget.

Some other i18n features should be added later, but those are basic for Web site i18n.

[Marc Garcia]

Initial patch (partial)

[Marc Garcia]

V1 patch includes all features except form i18n. Some parts are ugly and need to be coded better, it's just a starting point to see what needs to be changed.

I set milestone to 1.0 beta, because I think it should be included (if finished when 1.0-beta) on 1.0, so it's essential for djangonauts from many countries (most except the US, UK and Australia ;)

[Alex Gaynor]

Would it be possible to handle this as individual smaller entities, instead of one monolithic patch.

[Malcolm Tredinnick]

Yes, we have to do this, but this isn't the right approach, since there's too much manual work. Also, there are tickets already open for most of these things, from memory.

What I want to do here is auto-generate all of this information from the Local Data Markup Language files from the Unicode group. It should be pretty easy to do and then we just drop stuff in the appropriate locale directories. Django can ship a script to parse the file for a new language and we can make that script rely on one download (since it's only needed for maintainers, Django doesn't have to ship it).

I'll leave this open until I hunt out all the dupes later on tonight. I agree we should do this pre-1.0, Marc, but lots more translation stuff like this is probably a burden we can avoid.

[Marc Garcia]

First draft of a script for importing data from CLDR to django

[Marc Garcia]

Attached import_cldr.py isn't finished, and isn't working ok. I just uploaded it for people that is working on improve django's i18n.

[Malcolm Tredinnick]

This will have to wait until after 1.0. There's still too much work required to get it in before the beta and it requires enough internal changes that it will be potentially destabilising.

I still don't like the first approach in the patch because it adds a lot of settings and makes assumptions such as i18n being enabled. So even it would need some work (and not be backwards compatible with a more general solution). So, sorry, Marc, but this just came a bit late in the process. It's very important, but not worth the risk at the moment.

We'll definitely get it in by 1.1, though.

[Marc Garcia]

I absolutely agree.

Sadly I realized on that when improving and merging the existing patches.

The big news are that it'll be another big fucking party for Django 1.1 if it includes it. ;)

[mrts]

Imminent duplicated effort alert: ​http://babel.edgewall.org/ . Can't we just depend on Babel? Also, when things get refactored -- contrib.localflavour should really not be part of Django trunk.

[Malcolm Tredinnick]

Please do not post random unrelated comments on tickets (localflavour has nothing to do with this ticket). We aren't duplicating all of Babel, obviously, since we're well aware of that project, and we aren't about to have Django depending on a third-party package the size of Babel. We are incorporating some necessary minimal stuff so that Django's continues to have batteries included.

[Jacob]

#6974 was another duplicate.

[(none)]

Milestone post-1.0 deleted

[Marc Garcia]

Fixed in branches/soc2009/i18n-improvements.

[Marc Garcia]

diff of all work developed on the [soc2009/i18n]

[Marc Garcia]

diff of all work developed on the [soc2009/i18n] with .po and formats files removed

[miracle2k]

Not sure what the right place to report this is.

I'm given the soc2009/i18n-improvements branch a try, but have been running into the problem that my German locale is not being used in form submissions. I'm using de_DE as my LANGUAGE_CODE setting, and as far as I was able to tell this is because in django/utils/formats.py, it attempts to import the format file based on django.utils.translation.get_language(), which in my case would return "de-de", when in fact the module to import from is "django.conf.locale.de".

Loading the .mo files doesn't have the same problem, since the gettext loading mechanism seems to auto-fallback to the available "de" locale.

[miracle2k]

Another bug in that branch: floatformat doesn't correctly print the sign of negative values. Quick-fix here:

​http://github.com/miracle2k/django/commit/9259cb9bd1f40167a67714fc037d504c1247d979

[Jannis Leidel]

Replying to miracle2k:

Another bug in that branch: floatformat doesn't correctly print the sign of negative values. Quick-fix here:

​http://github.com/miracle2k/django/commit/9259cb9bd1f40167a67714fc037d504c1247d979

Thanks, I've fixed this in my current working branch: ​http://github.com/jezdez/django/tree/i18n

[Jannis Leidel]

Replying to miracle2k:

Not sure what the right place to report this is.

I'm given the soc2009/i18n-improvements branch a try, but have been running into the problem that my German locale is not being used in form submissions. I'm using de_DE as my LANGUAGE_CODE setting, and as far as I was able to tell this is because in django/utils/formats.py, it attempts to import the format file based on django.utils.translation.get_language(), which in my case would return "de-de", when in fact the module to import from is "django.conf.locale.de".

Loading the .mo files doesn't have the same problem, since the gettext loading mechanism seems to auto-fallback to the available "de" locale.

Good catch, I'll work on a fix for this. It probably involves a simple <locale>[:2] check in case the long format is not found.

[Jannis Leidel]

Updated patch with relevant parts of i18n GSoC

[Jannis Leidel]

Fixed issue with LANGUAGE_CODE of sublanguages

[Jannis Leidel]

Updated format discovery to also look in main locales if not found in sublocales formats

[Jannis Leidel]

Added Serbian and Serbian latin format strings from #11637

[Jannis Leidel]

More deprecation warnings and Renamed USE_FORMAT_I18N setting to USE_L10N

[Jannis Leidel]

Mainly updates to docs

[Jannis Leidel]

(In [11964]) Fixed #7980 - Improved i18n framework to support locale aware formatting (dates and numbers) and form processing.

Thanks to Marc Garcia for working on this during his Google Summer of Code 2009!

Additionally fixes #1061, #2203, #3940, #5526, #6449, #6231, #6693, #6783, #9366 and #10891.

[Jacques Beaurain <jacques.beaurain@…>]

I just created ticket #12550 with a patch for a bug caused by the changes comitted for this one. Since my patch only deals with DecimalField and this is a large submission I am not sure if there might be other types affected. I think it might be good for someone with more knowledge of the django.contrib.admin codebase to review this.

[Ramiro Morales]

This ticket was closed because the original fature has been implemented.

If you opened another ticket, there is no need to reopen this one. Doing so isn't the right way to bring attention to the issue.

[Karen Tracey]

As Ramiro said, we only need one ticket to track the follow-on bug, and the right ticket for that is the new one, not this one. Concerns about there possibly being other field types affected should be noted there. Tests covering all field types would be a way to find out if the problem affects other field types. A test needs to be developed anyway for the DecimalField case; it should not be hard to expand that test to also test other field types. That would be useful both to find out whether the problem affects other fields, and to put in the test suite in either case to ensure this kind of problem, for any field type, does not recur with any future code changes.

[Jacques Beaurain <jacques.beaurain@…>]

Sorry about that, my aim was not really to try and draw attention to that ticket, I already added tests and fixed it so that it does not happen for other field types.

I added notes here more to notify whoever was involved in this one that there might be other places (not related to admin list display necessarily) that was affected by this change and for anybody in the know to review the patch for this again.

[Jacob]

Milestone 1.2 deleted