Changes between Initial Version and Version 2 of Ticket #28231


Ignore:
Timestamp:
May 23, 2017, 12:09:21 AM (7 years ago)
Author:
Nir Izraeli
Comment:

Legend:

Unmodified
Added
Removed
Modified
  • Ticket #28231

    • Property Cc nirizr@… added
    • Property Summary bulk_create: avoid iterating `objs` more than necessary when `bulk_size` is providedbulk_create: avoid iterating `objs` more than necessary when `batch_size` is provided
  • Ticket #28231 – Description

    initial v2  
    1 When `bulk_size` provided in `bulk_create`, a user might assume (As I myself have) that `objs` iterable will not be iterated more than `bulk_size` times at once, and that no more than roughly `bulk_size` Model objects reside in memory at any given time.
     1When `batch_size` provided in `bulk_create`, a user might assume (As I myself have) that `objs` iterable will not be iterated more than `batch_size` times at once, and that no more than roughly `batch_size` Model objects reside in memory at any given time.
    22
    33When using `bulk_create` for relatively big sets of objects provided by a generator object, it would be prefered to avoid iterating over the entire generator object. Moreover, if not iterating over the generator object is deemed unnecessary or out-of-scope for django it would be prefered to make a comment on said behavior in documentation.
     
    55I suggest two possible solutions:
    661. Document this behavior (`bulk_create` converts passed `objs` iterator to a list), or
    7 2. Avoid doing so when `bulk_size` is given (or default is other than `None`. i.e. sqlite).
     72. Avoid doing so when `batch_size` is given (or default is other than `None`. i.e. sqlite).
    88
    99I did not research the possibility of avoiding the list conversion, but if that solution is accepted by the community I volunteer to investigate further and claim this ticket.
Back to Top