#1443 closed defect (fixed)
No support for dates before 1900
Reported by: | Antti Kaihola | Owned by: | Adrian Holovaty |
---|---|---|---|
Component: | Core (Other) | Version: | dev |
Severity: | normal | Keywords: | date 1900 datetime |
Cc: | rafaelbm_sbo@…, gary.wilson@…, brian.mabry.edwards@…, greenwaldjared@…, simon@… | Triage Stage: | Accepted |
Has patch: | yes | Needs documentation: | no |
Needs tests: | no | Patch needs improvement: | no |
Easy pickings: | no | UI/UX: | no |
Description
See logviewer for a short discussion.
#1070 changed date validation so dates before 1900 raise an error. But that's only to avoid problems further down the road.
>>> datetime(1805,12,24).strftime('%Y-%m-%d') ValueError: year=1805 is before 1900; the datetime strftime() methods require year >= 1900
Using PHP-style date formatting is broken too:
>>> dateformat.format(datetime(1899,4,12), 'Y-m-d') /usr/local/lib/djangomr2458/django/utils/tzinfo.py in _isdst(self, dt) ---> 50 stamp = time.mktime(tt) ValueError: year out of range
The problem is deeper in Python (or C, or the OS, or whatever):
>>> time.mktime((1901,12,14,0,0,0,0,0,0)) -2147477992.0 >>> time.mktime((1901,12,13,0,0,0,0,0,0)) OverflowError: mktime argument out of range >>> time.mktime((1899,12,31,0,0,0,0,0,0)) ValueError: year out of range
Would mx.DateTime be a solution?
>>> mx.DateTime.Date(1543,12,24).strftime('%Y-%m-%d') '1543-12-24'
I realize that adding another external module dependency is not a Good Thing. Maybe it's worth checking if mx.DateTime is similar enough to be used as a drop-in replacement for datetime in case it's installed on the system? This way those who need to go back to 19th century or before could just install mx.DateTime and enjoy.
Attachments (6)
Change History (35)
comment:1 by , 19 years ago
Resolution: | → wontfix |
---|---|
Status: | new → closed |
comment:2 by , 18 years ago
Cc: | added |
---|---|
Resolution: | wontfix |
Status: | closed → reopened |
Version: | → SVN |
I don't think that we necessarily need to use mx.DateTime
. At least we could fix the admin interface by not using strftime when saving values to the database. datetime.datetime
has no problem creating dates before 1900.
by , 18 years ago
Attachment: | datetime_pre_1900.patch added |
---|
patch for handling of pre-1900 strftime (with tests)
comment:3 by , 18 years ago
Summary: | no support for dates before 1900 → [patch] no support for dates before 1900 |
---|
by , 18 years ago
Attachment: | datetime_pre_1900.2.patch added |
---|
updated for recent change to date validator
comment:4 by , 18 years ago
Keywords: | date 1900 datetime added |
---|---|
Triage Stage: | Unreviewed → Accepted |
Marked as Accepted as I do think dates prior to 1900 will be an issue in a non-trivial proportion of applications. It's unfortunate that Python drops the ball here, and this adds another layer of complexity but the version here by SmileyChris doesn't require the mx.DateTime addition, and appears to handle everything.
Some issues prior to check in though:
Are there any potential backwards compat. issues here?
Can we directly replace datetime with datetime_pg (& there are datetime calls all over the code)?
Is this going to be a performance issue?
Alternatively - is it possible to get users who need these date ranges to install mx.DateTime (e.g. use the python version by default, and override with this/mx.DateTime on a user-setting basis)?
comment:5 by , 18 years ago
Cc: | added |
---|
comment:6 by , 18 years ago
Cc: | added |
---|
I'm also having problems with dates before 1900, using the current SVN version and newforms:
Traceback (most recent call last): File "C:\Python25\lib\site-packages\django\core\handlers\base.py" in get_response 77. response = callback(request, *callback_args, **callback_kwargs) File "C:\Documents and Settings\Usu�rio\Desktop\projeto1\exemplos\projeto1\cadastro\views.py" in lista_cliente 37. form.save() File "C:\Python25\lib\site-packages\django\newforms\models.py" in save 56. return save_instance(self, instance, fields, fail_message, commit) File "C:\Python25\lib\site-packages\django\newforms\models.py" in save_instance 35. instance.save() File "C:\Python25\lib\site-packages\django\db\models\base.py" in save 217. db_values = [f.get_db_prep_save(f.pre_save(self, False)) for f in non_pks] File "C:\Python25\lib\site-packages\django\db\models\fields\__init__.py" in get_db_prep_save 496. value = value.strftime('%Y-%m-%d') ValueError at /cadastro/clientes/8/ year=1000 is before 1900; the datetime strftime() methods require year >= 1900
comment:7 by , 17 years ago
Since you're just overriding a couple of (broken) portions of the core implementation, I don't really see any performance or compatibility issues here. But that's me.
comment:8 by , 17 years ago
On the other hand, it might be better to just put the robust strptime/strftime implementations in a support library, and make sure that Django uses those for all datetime formatting. I'll check with the sprint IRC folks.
follow-up: 12 comment:10 by , 17 years ago
The #3306 approach is more of a hack, though; it just fakes its way around the C time limitations, in order to support 1900-9999. But that doesn't help; I have dates between 1500 and 1900 in my databases... :-/
No time to dig deeper into this right now, but I'm now convinced that putting Dalke's code in a support library, and making sure to use that whenever the Django core converts dates, is the right solution. Introducing a Django-specific datetime type/module may be elegant, but it's also rather fragile if you have a view that gets data from non-Django sources, and want to render it through Django.
I can check the license with Andrew and adapt Chris' patch, if there's support for that approach.
comment:13 by , 17 years ago
Replying to jbronn:
@Fredrik: +1 (I need this support as well)
me too.. I'd really like to see this fixed
years before 1900 is something that python/django should support by default imho
is this platform specific btw? (i've read that it's an issue on osx for example)
thanks
comment:14 by , 17 years ago
Triage Stage: | Accepted → Ready for checkin |
---|
tests are working, we're all good.
comment:15 by , 17 years ago
Resolution: | → fixed |
---|---|
Status: | reopened → closed |
Triage Stage: | Ready for checkin → Design decision needed |
PhiR - I'm actually not that keen on that implementation. I can see some updates I could make to my original patch (wow, that was a long time ago) and still think it's a better way. Could you bring it up on django-dev please?
comment:17 by , 17 years ago
Cc: | added |
---|
comment:18 by , 17 years ago
Patch needs improvement: | set |
---|
FYI, I tried to apply both the datetime_pre_1900.2.patch and the datetime_pre_1900.2.2.patch and both generate a bunch of rejected diffs. I tried to hand-patch the rejects from 1900.2, but when I go into the admin site and edit an entry to change the date to before 1900, I get a ValueError exception saying "year=1786 is before 1900; the datetime strftime() methods require year >= 1900". I'm guessing that its just really hard to patch a moving target and that trunk has just progressed too far from when these patches were generated to be valid now.
I'm not terribly knowledgeable about the internals of django beyond the basics. I don't really have the time to study the code to get the proper expertise needed to truly patch this into current trunk (as it is this simple site I'm working on is taking up most of what little free time I have). I'm certainly willing to apply whatever patch someone can generate (SmileyChris or whoever) and report back my findings. My code is not terribly sophisticated and I'm really just needing this to be able to store basic date information for some historical data (cemetery records, actually), so if there are specific test that people want performed, that information would be good to know about as well.
Thanks in advance.
comment:19 by , 17 years ago
Also, just to be clear. I didn't apply both patches to the same tree (though I'm not entirely sure that would be bad since 1900.2.2 is supposed to just additionally fix oldforms, but I digress...). I re-checked-out trunk between applying 1900.2.2 and then subsequently applying 1900.2 and then proceeding to attempt to hand-patch.
Sorry for the spam, but just wanted to clarify that.
comment:20 by , 17 years ago
Patch needs improvement: | unset |
---|---|
Summary: | [patch] no support for dates before 1900 → No support for dates before 1900 |
Ok, so my naming sucked - the new module is called datetime_safe
. The methods that you'll normally use are just datetime_safe.new_date
and datetime_safe.new_datetime
.
It should handle all form (old and new) and database saving fine. Django only uses a datetime_safe object when it needs to, so if you're doing your own strftime
calls on pre-1900 dates, you'll need to convert the object yourself.
comment:21 by , 17 years ago
Owner: | changed from | to
---|---|
Status: | reopened → new |
comment:22 by , 17 years ago
Just tried the new patch. Applied cleanly to current trunk (rev 7407).
I was able to change the dates in my objects to pre-1900 dates via the admin interface without error. I was also able to retrieve the data via my previously-existing (generic) view code - think object_detail - without error.
If anyone can come up with other tests that should be performed, I'll oblige whatever I can.
comment:23 by , 17 years ago
Cc: | added |
---|
The way newforms treats < 1900 dates, with or without mx.DateTime, is another consideration for this ticket.
See also #7014 for a patch with tests to raise ValidationError instead of ValueError for dates before 1900 on newforms DateField/DateTimeFields.
comment:24 by , 17 years ago
Cc: | added |
---|
comment:25 by , 17 years ago
patch no longer cleanly applies (looks like a whitespace issue), and it fails unit tests:
Error while importing datetime_safe: File "./runtests.py", line 130, in django_tests mod = load_app(model_label) File "/Users/dan/src/django/tests/django/db/models/loading.py", line 72, in load_app mod = __import__(app_name, {}, {}, ['models']) ImportError: No module named datetime_safe
would it help for me to reroll this, or does SmileyChris have this under control?
comment:27 by , 16 years ago
Owner: | changed from | to
---|---|
Status: | new → assigned |
Triage Stage: | Design decision needed → Accepted |
comment:28 by , 16 years ago
Cc: | removed |
---|
comment:29 by , 16 years ago
Resolution: | → fixed |
---|---|
Status: | assigned → closed |
I'm marking this as a wontfix for now, because adding support for mx.DateTime would introduce too much complexity.