Opened 8 years ago

Closed 8 years ago

#27475 closed Bug (fixed)

Bug in admin with date_hierarchy and Daylight savings

Reported by: Hugo Tácito Owned by: nobody
Component: contrib.admin Version: dev
Severity: Normal Keywords:
Cc: desecho@… Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

When I have a date_hierarchy in admin and I select a range with a record with the day equal the day of a daylight savings I got an error of NonExistentTimeError.

I discovered that indeed 2016-10-16 00:00:00 does not exist in my Timezone ('America/Sao_Paulo') because that was when Daylight Savings started.

So when I have a record with that date in my database, it will brake the date_hierarchy feature.

Attachments (1)

djtest.zip (14.3 KB ) - added by Anton Samarchyan 8 years ago.

Download all attachments as: .zip

Change History (12)

comment:1 by Tim Graham, 8 years ago

I'm not sure what the expected behavior is. What could Django do if you have bad data?

comment:2 by Hugo Tácito, 8 years ago

Hello Tim,

I'm sorry for my lack of information and clarity, but even if I do have an existing datetime the problem still occur.

For example I have a User with date_joined equals 2016-10-16 12:17:44.270221-03 and when I select a date_hierarchy filter, that still raise the same exception of:

[12/Nov/2016 13:28:10] "GET /admin/auth/user/?date_joined__year=2016 HTTP/1.1" 200 6265
Internal Server Error: /admin/auth/user/
Traceback (most recent call last):
  File "/Users/hugo/.virtualenvs/suap/lib/python2.7/site-packages/django/core/handlers/base.py", line 174, in get_response
    response = self.process_exception_by_middleware(e, request)
  File "/Users/hugo/.virtualenvs/suap/lib/python2.7/site-packages/django/core/handlers/base.py", line 172, in get_response
    response = response.render()
  File "/Users/hugo/.virtualenvs/suap/lib/python2.7/site-packages/django/template/response.py", line 160, in render
    self.content = self.rendered_content
  File "/Users/hugo/.virtualenvs/suap/lib/python2.7/site-packages/django/template/response.py", line 137, in rendered_content
    content = template.render(context, self._request)
  File "/Users/hugo/.virtualenvs/suap/lib/python2.7/site-packages/django/template/backends/django.py", line 95, in render
    return self.template.render(context)
  File "/Users/hugo/.virtualenvs/suap/lib/python2.7/site-packages/django/template/base.py", line 206, in render
    return self._render(context)
  File "/Users/hugo/.virtualenvs/suap/lib/python2.7/site-packages/django/template/base.py", line 197, in _render
    return self.nodelist.render(context)
  File "/Users/hugo/.virtualenvs/suap/lib/python2.7/site-packages/django/template/base.py", line 992, in render
    bit = node.render_annotated(context)
  File "/Users/hugo/.virtualenvs/suap/lib/python2.7/site-packages/django/template/base.py", line 959, in render_annotated
    return self.render(context)
  File "/Users/hugo/.virtualenvs/suap/lib/python2.7/site-packages/django/template/loader_tags.py", line 173, in render
    return compiled_parent._render(context)
  File "/Users/hugo/.virtualenvs/suap/lib/python2.7/site-packages/django/template/base.py", line 197, in _render
    return self.nodelist.render(context)
  File "/Users/hugo/.virtualenvs/suap/lib/python2.7/site-packages/django/template/base.py", line 992, in render
    bit = node.render_annotated(context)
  File "/Users/hugo/.virtualenvs/suap/lib/python2.7/site-packages/django/template/base.py", line 959, in render_annotated
    return self.render(context)
  File "/Users/hugo/.virtualenvs/suap/lib/python2.7/site-packages/django/template/loader_tags.py", line 173, in render
    return compiled_parent._render(context)
  File "/Users/hugo/.virtualenvs/suap/lib/python2.7/site-packages/django/template/base.py", line 197, in _render
    return self.nodelist.render(context)
  File "/Users/hugo/.virtualenvs/suap/lib/python2.7/site-packages/django/template/base.py", line 992, in render
    bit = node.render_annotated(context)
  File "/Users/hugo/.virtualenvs/suap/lib/python2.7/site-packages/django/template/base.py", line 959, in render_annotated
    return self.render(context)
  File "/Users/hugo/.virtualenvs/suap/lib/python2.7/site-packages/django/template/loader_tags.py", line 69, in render
    result = block.nodelist.render(context)
  File "/Users/hugo/.virtualenvs/suap/lib/python2.7/site-packages/django/template/base.py", line 992, in render
    bit = node.render_annotated(context)
  File "/Users/hugo/.virtualenvs/suap/lib/python2.7/site-packages/django/template/base.py", line 959, in render_annotated
    return self.render(context)
  File "/Users/hugo/.virtualenvs/suap/lib/python2.7/site-packages/django/template/loader_tags.py", line 69, in render
    result = block.nodelist.render(context)
  File "/Users/hugo/.virtualenvs/suap/lib/python2.7/site-packages/django/template/base.py", line 992, in render
    bit = node.render_annotated(context)
  File "/Users/hugo/.virtualenvs/suap/lib/python2.7/site-packages/django/template/base.py", line 959, in render_annotated
    return self.render(context)
  File "/Users/hugo/.virtualenvs/suap/lib/python2.7/site-packages/django/template/library.py", line 223, in render
    _dict = self.func(*resolved_args, **resolved_kwargs)
  File "/Users/hugo/.virtualenvs/suap/lib/python2.7/site-packages/django/contrib/admin/templatetags/admin_list.py", line 380, in date_hierarchy
    } for day in days]
  File "/Users/hugo/.virtualenvs/suap/lib/python2.7/site-packages/django/db/models/query.py", line 258, in __iter__
    self._fetch_all()
  File "/Users/hugo/.virtualenvs/suap/lib/python2.7/site-packages/django/db/models/query.py", line 1074, in _fetch_all
    self._result_cache = list(self.iterator())
  File "/Users/hugo/.virtualenvs/suap/lib/python2.7/site-packages/django/db/models/query.py", line 158, in __iter__
    for row in compiler.results_iter():
  File "/Users/hugo/.virtualenvs/suap/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 808, in results_iter
    row = self.apply_converters(row, converters)
  File "/Users/hugo/.virtualenvs/suap/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 792, in apply_converters
    value = converter(value, expression, self.connection, self.query.context)
  File "/Users/hugo/.virtualenvs/suap/lib/python2.7/site-packages/django/db/models/expressions.py", line 950, in convert_value
    value = timezone.make_aware(value, self.tzinfo)
  File "/Users/hugo/.virtualenvs/suap/lib/python2.7/site-packages/django/utils/timezone.py", line 364, in make_aware
    return timezone.localize(value, is_dst=is_dst)
  File "/Users/hugo/.virtualenvs/suap/lib/python2.7/site-packages/pytz/tzinfo.py", line 327, in localize
    raise NonExistentTimeError(dt)
NonExistentTimeError: 2016-10-16 00:00:00

Even with an existing datetime.

The admin I did to test:

class UserAdmin(admin.ModelAdmin):
    date_hierarchy = 'date_joined'
    list_display = ('email', 'first_name', 'last_name', 'date_joined')
    list_filter = ('is_staff', 'is_superuser')


admin.site.unregister(User)
admin.site.register(User, UserAdmin)

The internationalization settings:

LANGUAGE_CODE = 'pt-br'

TIME_ZONE = 'America/Sao_Paulo'

USE_I18N = True

USE_L10N = True

USE_TZ = True

With TIME_ZONE = 'UTC' the problem doesn't occur.

Edit:
I've did this in a new clean project, with just this object at the User model.

Version 2, edited 8 years ago by Hugo Tácito (previous) (next) (diff)

comment:3 by Hugo Tácito, 8 years ago

This also happens if we set a datetime to 2015-10-18. For example: 2015-10-18 10:45:29.270221-03
In 2015 at 10-18 was started the daylight savings at Brazil too.

https://www.timeanddate.com/time/change/brazil/sao-paulo?year=2015

comment:4 by Tim Graham, 8 years ago

Any idea what Django should do to resolve this?

comment:5 by Hugo Tácito, 8 years ago

Hello Tim,

It seems that the datetime truncation is too hard in the date_hierarchy templatetag.

Changing this line:
https://github.com/django/django/blob/master/django/contrib/admin/templatetags/admin_list.py#L385

to:
days = getattr(days, dates_or_datetimes)(field_name, 'second')

Probably will fix it.
I can also create a patch if you want.

comment:6 by Aymeric Augustin, 8 years ago

Triage Stage: UnreviewedAccepted

The problem seems to stem from the fact that Django wants dates, but fetches them as datetimes, and applies the converter for datetimes, which causes an exception in that case.

It isn't possible to represent "2016-10-16 in local time in São Paulo" as datetime.datetime(2016, 10, 16, tzinfo=pytz.timezone('America/Sao_Paulo')) because that value doesn't exist. The only proper way to represent it is datetime.date(2016, 10, 16).

I believe the resolution should be along the lines of making the year / month / day truncation expressions cast their result to a DateField instead of a DateTimeField. Reducing the problem to a lower-level test case may help.

It seems possible that the same problem could occur with the hour truncation expression if some time zones have their DST between 1:30 and 2:30. I'm not sure what we could do about that; it seems much harder to fix.

by Anton Samarchyan, 8 years ago

Attachment: djtest.zip added

comment:7 by Anton Samarchyan, 8 years ago

Cc: desecho@… added
Version: 1.10master

I've attached a django project to reproduce the problem. Admin credentials - admin/AiNeuw3uOhvei7da
To reproduce, open /admin/app/test/?date__month=10&date__year=2016

Last edited 8 years ago by Tim Graham (previous) (diff)

comment:8 by Anton Samarchyan, 8 years ago

Has patch: set

Added PR

comment:9 by Tim Graham, 8 years ago

Patch needs improvement: set

comment:10 by Anton Samarchyan, 8 years ago

Patch needs improvement: unset

comment:11 by Tim Graham <timograham@…>, 8 years ago

Resolution: fixed
Status: newclosed

In e88d2df:

Fixed #27475 -- Fixed NonExistentTimeError crash in ModelAdmin.date_hierarchy.

Note: See TracTickets for help on using tickets.
Back to Top