Admin date_hierarchy filter by month displays an extra day at timezone boundary.

[Lavrenov Ivan]

When I authorized by user with not-UTC timezone, like America/Los_Angeles , and open filter by date in month, I see one extra day, that follows to the first day of the previous month

[Lavrenov Ivan]

representation of bug

[Tim Graham]

Could you be more specific about how to reproduce this? I thought you meant TIME_ZONE = 'Los-Angeles/America' but I get "Incorrect timezone setting". Which database are you using?

[Lavrenov Ivan]

The timezone is America/Los_Angeles, sorry for inaccuracy.
I'm using Mysql database. I think it is something сonnected with converstion to local time. I have a record, which in UTC in the 1st September, but in local time it is still in the 31 August.

P.S. Values stored in UTC format

[Lavrenov Ivan]

Replying to Tim Graham:

Could you be more specific about how to reproduce this? I thought you meant TIME_ZONE = 'Los-Angeles/America' but I get "Incorrect timezone setting". Which database are you using?

The timezone is America/Los_Angeles, sorry for inaccuracy.
I'm using Mysql database. I think it is something сonnected with converstion to local time. I have a record, which in UTC in the 1st September, but in local time it is still in the 31 August.
P.S. Values stored in UTC format

[Carlton Gibson]

Yes. I can reproduce this.

1. Have a DateTimeField in date_hierarchy (and optionally `list_display)
2. Create an object in the DB for midnight 1st Sep (UTC).
3. Set project TIME_ZONE = 'America/Los_Angeles'

• Test object is show in list at Aug 31 5PM.
• Date hierarchy choice is created for 1st September
• Link for that is incorrect: ?<field>__day=1&<field>__month=8&<field>__year=2018, i.e. Aug 1st.

Presumably error is in ​`date_hierarchy()` template tag function.

[Alexander Holmbäck]

​PR

[Carlton Gibson]

Duplicate reported in #30749

[yab]

Two distinct date_hierarchy bugs : one relate to TZ effects and one related to DST effects :

#29724 (TZ matter)
It may be solved modifying django/contrib/admin/templatetags/admin_list.py using queryset.filter (instead of queryset.dates) and lists (that may be slow) :

from django.utils.timezone import make_naive
from django.conf import settings

and in def link(filters):

		elif year_lookup and month_lookup:
			### This is a way to avoid using queryset.dates fucntion, which was giving naive datetimes with problems when the monthrange is different between utc and localtime.
			month_filter=field_name+'__month'
			year_filter=field_name+'__year'
			dates_or_datetimes = cl.model.objects.filter(**{year_filter:year_lookup, month_filter:month_lookup}).values_list(field_name, flat=True)
			days = dates_or_datetimes
			if isinstance(dates_or_datetimes[0], datetime.datetime) and settings.USE_TZ:
				day_list = []
				days = []
				for day in dates_or_datetimes:
					if make_naive(day).day not in day_list:
						day_list.append(make_naive(day).day)
						days.append(make_naive(day))
			

###30749 (DST matter)
In Europe/Paris timezone, on the October DST change, the from_date + timedelta(days=32)).replace(day=1) used in django/contrib/admin/views/main.py to get next month's first day does not select objects of October 31th. They are not shown in the change list when October is selected through date_hierarchy. (steps to reproduce : sqlite, DateField, USE_TZ, TIMEZONE='Europe/Paris', Object date 2019-10-31 or 2018-10-31...).
It seems to be resolved by adding make_aware and tzinfo=None to to_date :
django/contrib/admin/views/main.py:

				elif month:

					to_date = (from_date + timedelta(days=32)).replace(day=1)
					if settings.USE_TZ:
						to_date = make_aware(to_date.replace(tzinfo=None))

[Carlton Gibson]

OK, thanks for the follow-up yab. TBH I'm still inclined to view these as the same issue: incorrect collection of items at time-period boundaries (demonstrated with DST and TZ examples). However... one-ticket/two-tickets is not so important.

More is that, we need to make sure we have test cases for both these types of example, and then we can make sure any fix addresses both of those.

Super. 👍

[yab]

OK, my proposal passes existing tests. Not sure how to create specific test cases for that purpose. My proposal is above and there : ​https://github.com/yab228/django/blob/master/django/contrib/admin/templatetags/admin_list.py

[Carlton Gibson]

yab: please make a pull request with your suggested changes. (See the ​contributing guide.)

For tests, please add a test case with the example(s) here. Presumably they should fail with the exisiting code, showing the bug, and then pass with your patch applied.

[yab]

The tests I made at home were OK.
But several of the automatic same tests on github (in djangoci) were not OK. Especially test_date_hierarchy_timezone_dst.
Thoses automatic tests in many environments are too hard for a newbie like me. Too much time that I have not. I am giving up.

One last thing before I vanish : do the django programers plan to modify the hole behaviour of date_hierarchy ? I read it should stick more to result_list... If you can "just" tell me where I can see the discussions about that, I'm interested in reading. It seems to me date_hierarchy has to have two branches, on in case of datetimes+TZ_USE, one in other cases.

Thanks, anyway, to all the django-project team.

[Carlton Gibson]

#30922 was also a duplicate related.

[Hasan Ramezani]

​PR

[Mariusz Felisiak <felisiak.mariusz@…>]

In 53b6a466:

Refs #29724 -- Added is_dst parameter to QuerySet.datetimes().

Thanks Simon Charette for the review and Mariusz Felisiak for tests.

[Mariusz Felisiak <felisiak.mariusz@…>]

In 55cdf6c5:

Fixed #29724 -- Fixed timezone handling in ModelAdmin.date_hierarchy queries.

Thanks Alexander Holmbäck for the initial patch.