Opened 9 hours ago

Last modified 53 minutes ago

#36088 assigned Cleanup/optimization

Avoid unnecessary DEFAULT usage on bulk_create for models with db_default fields — at Version 2

Reported by: Simon Charette Owned by: Simon Charette
Component: Database layer (models, ORM) Version: dev
Severity: Normal Keywords: unnest insert db_default default bulk_create
Cc: Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description (last modified by Simon Charette)

When bulk_create is used for models composed of fields with db_default the resulting INSERT statement use DEFAULT placeholders to signify that a field must use the database defined default.

For example, the following code

class Author(models):
    name = models.CharField(max_length=100)
    created_at = models.DateTimeField(db_default=Now())

Author.objects.bulk_create([Author(name="foo"), Author(name="bar")])

Will result in the following SQL

INSERT INTO author (name, created_at) VALUES (%s, DEFAULT), (%s, DEFAULT)

But in cases where no db_default is provided for all bulk-created instances there is no point in specifying DEFAULT for each row as that's what the database will do if the column is not specified at all. In other words the above SQL is equivalent to

INSERT INTO author (name) VALUES (%s), (%s)

but the latter query simplification provide benefits:

Firstly, it would allow the UNNEST optimization introduced in #35936 (a16eedcf9c69d8a11d94cac1811018c5b996d491) to be enabled for models that define db_default fields. Alas since DEFAULT is an expression and the optimization must be disabled in their presence no models making use of db_default can take advantage of it.

In other words, on Postgres, the SQL could be

INSERT INTO author (name) SELECT * FROM unnest([%s, %s])

which has demonstrated benefits.

Secondly, pruning the field would avoid having to provide the db_default expression for all model instances on backends that don't support DEFAULT in bulk-inserts such as Oracle and SQLite. In other words the following SQL would be avoided

INSERT INTO author (name, created_at) VALUES (%s, NOW()), (%s, NOW())

Lastly, it just make the query smaller as no DEFAULT has to be provided for each row for each columns with a defined db_default which surely reduce the parsing time on the backend.

Change History (2)

comment:1 by Simon Charette, 9 hours ago

Summary: Avoided unnecessary DEFAULT usage on bulk_create.Avoid unnecessary DEFAULT usage on bulk_create.

comment:2 by Simon Charette, 8 hours ago

Description: modified (diff)
Summary: Avoid unnecessary DEFAULT usage on bulk_create.Avoid unnecessary DEFAULT usage on bulk_create for models with db_default fields
Note: See TracTickets for help on using tickets.
Back to Top