BulkProcessMixin on models.Model

[Myung Eui Yoon]

New Feature for convinient context manager for bulk create/update.
Just inherit MoModelBulkProcessMixin on Model class.

        class BulkyModel(models.Model, ModelBulkProcessMixin):
            id = models.BigAutoField(primary_key=True)
            name = models.CharField(max_length=10, null=False)

        names = [f"name-{num}" for num in range(100_100_000)]

        # Case : Raw bulk_create
        objs = [BulkyModel(name=name) for name in names]
        BulkyModel.objects.bulk_create(
            objs
        )  # We should maintain big array size and this leades to OOM error

        # Case : Chunked bulk_create
        objs = list()
        for name in names:
            obj = BulkyModel(name=name)
            objs.append(obj)
            if len(objs) > 10_1000:
                BulkyModel.objects.bulk_create(objs)
                objs.clear()
        if len(objs) > 0:
            BulkyModel.objects.bulk_create(objs)
            objs.clear()

        # Case : With ModelBulkProcessMixin
        with BulkyModel.gen_bulk_create(batch_size=10_000) as bulk:
            for name in names:
                bulk.add(BulkyModel(name=name))

​https://github.com/django/django/pull/15578

[Myung Eui Yoon]

Issued Pull Request
​https://github.com/django/django/pull/15577

[Mariusz Felisiak]

Thanks for this patch, however I don't see how this (quite complicated) implementation optimize creating objects in bulk in a significant way. It's also not something that needs to be built into Django itself. It sounds like a third-party package is the best way to proceed.

Please ​follow the triaging guidelines with regards to wontfix tickets and take the idea to DevelopersMailingList to reach a wider audience and see what other think.