Opened 12 years ago

Closed 11 years ago

Last modified 11 years ago

#18766 closed Cleanup/optimization (fixed)

Models with datetime fields fail to save when date is below 1900 and USE_TZ is True

Reported by: Alexey Boriskin Owned by: Aymeric Augustin
Component: Documentation Version: dev
Severity: Normal Keywords: timezone, tz
Cc: Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

I have model with datetime field in it. When I save this model in admin site and the filled date is below 1900, I am getting the following traceback:

  File "/Users/voidus/Documents/workspace/django-trunk/django/contrib/admin/options.py", line 709, in save_model
    obj.save()
  File "/Users/voidus/Documents/workspace/django-trunk/django/db/models/base.py", line 524, in save
    force_update=force_update, update_fields=update_fields)
  File "/Users/voidus/Documents/workspace/django-trunk/django/db/models/base.py", line 595, in save_base
    rows = manager.using(using).filter(pk=pk_val)._update(values)
  File "/Users/voidus/Documents/workspace/django-trunk/django/db/models/query.py", line 573, in _update
    return query.get_compiler(self.db).execute_sql(None)
  File "/Users/voidus/Documents/workspace/django-trunk/django/db/models/sql/compiler.py", line 991, in execute_sql
    cursor = super(SQLUpdateCompiler, self).execute_sql(result_type)
  File "/Users/voidus/Documents/workspace/django-trunk/django/db/models/sql/compiler.py", line 813, in execute_sql
    sql, params = self.as_sql()
  File "/Users/voidus/Documents/workspace/django-trunk/django/db/models/sql/compiler.py", line 956, in as_sql
    val = field.get_db_prep_save(val, connection=self.connection)
  File "/Users/voidus/Documents/workspace/django-trunk/django/db/models/fields/__init__.py", line 303, in get_db_prep_save
    prepared=False)
  File "/Users/voidus/Documents/workspace/django-trunk/django/db/models/fields/__init__.py", line 834, in get_db_prep_value
    value = self.get_prep_value(value)
  File "/Users/voidus/Documents/workspace/django-trunk/django/db/models/fields/__init__.py", line 820, in get_prep_value
    if value is not None and settings.USE_TZ and timezone.is_naive(value):
  File "/Users/voidus/Documents/workspace/django-trunk/django/utils/timezone.py", line 271, in is_naive
    return value.tzinfo is None or value.tzinfo.utcoffset(value) is None
  File "/Users/voidus/Documents/workspace/django-trunk/django/utils/timezone.py", line 71, in utcoffset
    if self._isdst(dt):
  File "/Users/voidus/Documents/workspace/django-trunk/django/utils/timezone.py", line 89, in _isdst
    stamp = _time.mktime(tt)
ValueError: year out of range

I think that is because utcoffset and some other functions use _isdst function, which calls time.mktime and it have some limitation on the date range it can handle.

time module can handle dates in the more narrow range (1969-2038) then datetime module (1-9999).

Change History (20)

comment:1 by Alexey Boriskin, 12 years ago

As wikipedia article says, DST was not used before at least 1916. So, maybe simple check like

if dt.year < 1916:
    return False

will work

comment:2 by Aymeric Augustin, 12 years ago

Owner: changed from nobody to Aymeric Augustin

comment:3 by Alexey Boriskin, 12 years ago

The other corner case is dates after 2038. While simple check for < 1916 will work for dates before 1900, as there were no DST at that time, after 2038 we will probably still have DST. 2038 is not near, but I can imagine a calendar application with events in future.

comment:4 by Aymeric Augustin, 12 years ago

Indeed, this is a limitation of django.utils.tzinfo.LocalTimezone.

Could you try installing pytz (as strongly recommeded by the docs)? If that fixes the problem, I'd lean towards a documentation fix rather than extending the huge hack that is LocalTimezone.

Note that dates after 2038 are handled by returning incorrect data rather than crashing. I deeply dislike that, but I can't change it easily, for backwards-compatibility reasons.

Version 0, edited 12 years ago by Aymeric Augustin (next)

comment:5 by Aymeric Augustin, 12 years ago

See also #17262.

comment:6 by Aymeric Augustin, 12 years ago

Resolution: needsinfo
Status: newclosed

Waiting to know if installing pytz resolves the problem...

comment:7 by anonymous, 12 years ago

Yes, installing pytz resolves the problem.

comment:8 by Alexey Boriskin, 12 years ago

Sorry, that was me, was logged out somehow.

comment:9 by Aymeric Augustin, 12 years ago

Component: UncategorizedDocumentation
Resolution: needsinfo
Status: closedreopened
Triage Stage: UnreviewedAccepted
Type: BugCleanup/optimization

I'm accepting the ticket primarily as a documentation issue. This is related to #17262. I still have to decide how to deal with this.

comment:10 by Alexey Boriskin, 12 years ago

Maybe it would be OK to require pytz installed if user set USE_I18N = True in settings.py?
Or maybe we can even set a hard dependency? I think it's now commonly consented that django should make more use of pip - at least localflavor will be separated, etc. So, maybe pytz would be the first thing to require?

comment:11 by Aymeric Augustin, 12 years ago

USE_TZ = True is in the default settings files (for a good reason) and we want to keep Django usable without having to learn pip / virtualenv first (also for a good reason). I went to great lengths to ensure that pytz wouldn't be a mandatory.

I think it's more promising to raise a more explicit error in this case, ie. "value outside of supported range, please install pytz".

I

comment:12 by Alexey Boriskin, 12 years ago

Yeah, it may be a good solution. But may this behaviour be considered as a regression? Imagine, you have some values outside of supported range in database, then upgrade django one version upper, set USE_TZ=True, as docs insist. And live being happy until someday you see in logs such message about pytz.

comment:13 by Aymeric Augustin, 12 years ago

Status: reopenednew

comment:14 by Matthew Somerville, 11 years ago

Just to back up void's comment, I have just upgraded a site from Django 1.1 to 1.5. One of the few things that only bit me once it went live (and thanks as ever for the good documentation on upgrades and consistency :) ) was that I have some very old datetimes in the live database (back to the 17th century) that were working fine, that then started to throw "ValueError: year out of range" 500s due to this issue. It does indeed resolve if pytz is installed (or if USE_TZ is set back to False, which I've done for now).

comment:15 by Aymeric Augustin <aymeric.augustin@…>, 11 years ago

Resolution: fixed
Status: newclosed

In ded11aa620e5b38c78ff92e691a459d1977a2539:

Fixed #18766 -- Pointed to pytz when LocalTimezone fails.

Thanks void for the report.

comment:16 by Aymeric Augustin <aymeric.augustin@…>, 11 years ago

In 7c31e195db963ad1ebb28265e721097000fa84d1:

[1.6.x] Fixed #18766 -- Pointed to pytz when LocalTimezone fails.

Thanks void for the report.

Backport of ded11aa6 from master.

comment:17 by Aymeric Augustin <aymeric.augustin@…>, 11 years ago

In 1a1e14786aa35b4d80b46be747b5128800cf0f97:

Hardened the test introduced in ded11aa6. Refs #18766.

Inputs acceptable to time.mktime are platform-dependent.

comment:18 by Aymeric Augustin <aymeric.augustin@…>, 11 years ago

In 0035a0ce2ec243ff9255157117c20caf41a61284:

[1.6.x] Hardened the test introduced in ded11aa6. Refs #18766.

Inputs acceptable to time.mktime are platform-dependent.

Backport of 1a1e1478 from master.

comment:19 by Aymeric Augustin <aymeric.augustin@…>, 11 years ago

In c687bf0620d67e0579302515904ffe169581d036:

Further hardening. Refs #18766.

comment:20 by Aymeric Augustin <aymeric.augustin@…>, 11 years ago

In b085e7c3033d49e4d7525bacb1f34340730918a8:

[1.6.x] Further hardening. Refs #18766.

Backport of c687bf0 from master.

Note: See TracTickets for help on using tickets.
Back to Top