Evaluation of zero-length slices of queryset.values() fails

[Marc DEBUREAUX]

When using a QuerySet returning no values without a whole model behind, the Paginator fails to display data from page.

Example OK:

queryset = Model.objects.filter(my_filter_return_nothing=value)
paginator = Paginator(queryset, 10)
list(paginator.page(1))
>>> []

Example KO:

queryset = Model.objects.values('whatever_field').filter(my_filter_return_nothing=value)
paginator = Paginator(queryset, 10)
list(paginator.page(1))
>>>
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-38-56e5635092e7> in <module>()
----> 1 list(paginator.page(1))

\lib\site-packages\django\core\paginator.py in __len__(self)
    115
    116     def __len__(self):
--> 117         return len(self.object_list)
    118
    119     def __getitem__(self, index):

\lib\site-packages\django\db\models\query.py in __len__(self)
    238
    239     def __len__(self):
--> 240         self._fetch_all()
    241         return len(self._result_cache)
    242

\lib\site-packages\django\db\models\query.py in _fetch_all(self)
   1072     def _fetch_all(self):
   1073         if self._result_cache is None:
-> 1074             self._result_cache = list(self.iterator())
   1075         if self._prefetch_related_lookups and not self._prefetch_done:
   1076             self._prefetch_related_objects()

\lib\site-packages\django\db\models\query.py in __iter__(self)
    110         names = extra_names + field_names + annotation_names
    111
--> 112         for row in compiler.results_iter():
    113             yield dict(zip(names, row))
    114

\lib\site-packages\django\db\models\sql\compiler.py in results_iter(self, results)
    805         if results is None:
    806             results = self.execute_sql(MULTI)
--> 807         fields = [s[0] for s in self.select[0:self.col_count]]
    808         converters = self.get_converters(fields)
    809         for rows in results:

AttributeError: 'SQLCompiler' object has no attribute 'col_count'

Thanks.

[Tim Graham]

Is this a regression in Django 1.9? Can you provide a test for tests/pagination/tests.py?

[Sergey Fedoseev]

I can reproduce it on 1.9 and can't reproduce it on 1.8.7.

[Sergey Fedoseev]

Bisected to afe0bb7b13bb8dc4370f32225238012c873b0ee3.

Minimal example to reproduce:

from django.contrib.contenttypes.models import ContentType
list(ContentType.objects.values('model')[0:0])

[Marc DEBUREAUX]

Quick and dirty monkey patch.

# django monkey patch
# https://code.djangoproject.com/ticket/25894
try:
    from django.db.models.sql.compiler import SQLCompiler

    def _results_iter(self, results=None):
        if not hasattr(self, 'col_count'):
            self.col_count = 0
        if not self.select:
            self.select = []
        return self._results_iter(results)

    SQLCompiler._results_iter = SQLCompiler.results_iter
    SQLCompiler.results_iter = _results_iter
except:
    pass

May not work with everything.

[Sergey Fedoseev]

PR -- ​https://github.com/django/django/pull/5808

[Tim Graham <timograham@…>]

In 69b69f6d:

Fixed #25894 -- Fixed evaluation of zero-length slices of QuerySet.values().

[Tim Graham <timograham@…>]

In 192d1eb:

[1.9.x] Fixed #25894 -- Fixed evaluation of zero-length slices of QuerySet.values().

Backport of 69b69f6d6085d80ccf36a3e999e9e5b98d580786 from master

[Tim Graham]

The new test doesn't pass on Oracle.

======================================================================
ERROR: test_zero_length_values_slicing (queries.tests.WeirdQuerysetSlicingTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/tim/code/django/tests/queries/tests.py", line 2435, in test_zero_length_values_slicing
    self.assertQuerysetEqual(Article.objects.values()[n:n], [])
  File "/home/tim/code/django/django/test/testcases.py", line 933, in assertQuerysetEqual
    items = six.moves.map(transform, qs)
  File "/home/tim/code/django/django/db/models/query.py", line 258, in __iter__
    self._fetch_all()
  File "/home/tim/code/django/django/db/models/query.py", line 1072, in _fetch_all
    self._result_cache = list(self.iterator())
  File "/home/tim/code/django/django/db/models/query.py", line 112, in __iter__
    for row in compiler.results_iter():
  File "/home/tim/code/django/django/db/models/sql/compiler.py", line 792, in results_iter
    fields = [s[0] for s in self.select[0:self.col_count]]
AttributeError: 'SQLCompiler' object has no attribute 'col_count'

[Sergey Fedoseev]

On oracle this seems to be broken since 1.8.
PR -- ​https://github.com/django/django/pull/5834

[Tim Graham <timograham@…>]

In ed1bcf0:

Refs #25894 -- Fixed evaluation of zero-length slices of QuerySet.values() on Oracle.

[Tim Graham <timograham@…>]

In 14146f63:

[1.9.x] Refs #25894 -- Fixed evaluation of zero-length slices of QuerySet.values() on Oracle.

Backport of ed1bcf05158acf4bf4e0189d477b6c762bd0133e from master

[Marc DEBUREAUX]

It doesn't work with the following case:

from django.contrib.contenttypes.models import ContentType
list(ContentType.objects.values('model')[0:0])

Here the patch I use based on the last commit. Does I miss something else?

try:
    def _set_limits(self, low=None, high=None):
        self._set_limits(low, high)
        if self.low_mark == self.high_mark:
            self.set_empty()

    from django.db.models.sql.query import Query
    Query._set_limits = Query.set_limits
    Query.set_limits = _set_limits
except:
    pass

[Marc DEBUREAUX]

Nevermind, I forgot to delete lines from compiler.