Changes between Initial Version and Version 3 of Ticket #35904


Ignore:
Timestamp:
Dec 6, 2024, 4:28:36 AM (12 days ago)
Author:
JorisBenschop
Comment:

Legend:

Unmodified
Added
Removed
Modified
  • Ticket #35904

    • Property Summary Speed up fixture loading by bulk insertSpeed up fixture loading by adding options bulk insert/create
    • Property Type UncategorizedNew feature
    • Property Has patch set
  • Ticket #35904 – Description

    initial v3  
    1 I noticed that loading many fixtures is fairly inefficient, in that it tries an update and then an insert for each record. I would propose an additional option to the command line (—bulk-insert) that only runs the insert queries, but also detects if subsequent items in the fixtures are from the same model, so they may be inserted using bulk_insert. If there is genuine interest in this feature i will develop it and submit. I have this running in my own codebase already.
    2 Please also indicate if there is no interest, and i spend my energy elsewhere =)
     1 As per ​this forum discussion, I have created a patch to improve load times for the loaddata command under some circumstances.
     2
     3Currently the “loaddata” management command uses the obj.save() method for each deserialized object within a fixture. This function first tries an UPDATE statement and, if that fails, tries an INSERT statement. By using the --force_insert a reduction of 50% of queries is achieved.
     4
     5A second option is to use bulk_create for insertion of multiple records. This improves insertion speed by (n-1/n), or ~99% for insertion of 100 records.
     6
     7These options are not meant to cover each use case, and therefore are set to optional.
     8
     9
     10
Back to Top