QuerySet._fill_cache is not thread-safe

[mrts]

If several threads iterate over the same queryset, the following error occurs:

File "/some/path/__init__.py", line 624, in prerender
   for obj in objects: # <-- objects is a QuerySet

 File "/some/path/python2.5/site-packages/django/db/models/query.py", line 106, in _result_iter
   self._fill_cache()

 File "/some/path/python2.5/site-packages/django/db/models/query.py", line 692, in _fill_cache
   self._result_cache.append(self._iter.next())

ValueError: generator already executing

That happens as multiple threads concurrently both append to self._result_cahce and consume self._iter (see source:/django/trunk/django/db/models/query.py@11584#L692).

[mrts]

The question is how and whether this should be fixed, as using a shared queryset is not necessarily a good thing (it makes mostly sense to consume it with list() and store the list or use cache instead).

[mrts]

Summarized it as follows in the unofficial threadsafety wiki page: Django core is generally threadsafe as of 1.0.3 / 1.1. However, QuerySets are known not to be thread-safe, see #11906. Usually that does not pose problems as they are (or should be) not shared between threads in Django. The exception to that rule is the use of exotic global/class-level/shared instance variable querysets in your own code (e.g. when using the ORM outside of the Django dispatch system), where you are assumed to know what you are doing and protect them appropriately anyway.

[David Reynolds]

Just thought I'd weigh in and mention that I have seen this occurring too.

Does anyone have any idea on how we could go about fixing it?

[Jacob]

Milestone 1.3 deleted

[Aymeric Augustin]

Change UI/UX from NULL to False.

[Aymeric Augustin]

Change Easy pickings from NULL to False.

[Vlada Macek]

Just thought to tell here it happened to me today on a not very intensive site...

[Anssi Kääriäinen]

QuerySet _fill_cache is gone as part of QuerySet iterator behaviour fixing.