Opened 4 years ago

Last modified 4 years ago

#32542 closed Bug

Native postgres json field support is broken — at Version 1

Reported by: Artem Owned by: nobody
Component: Database layer (models, ORM) Version: 3.1
Severity: Normal Keywords:
Cc: Triage Stage: Unreviewed
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description (last modified by Artem)

In Django 3.1.1, there was a refactoring of encoders/decoders of DB fields which affected postgres json/jsonb field support.

Before:

Postgres json and jsonb field values are encoded/decoded into python native format by psycopg2. JSONField had it's own decoder but if was omitted if db has native json support.

After:

Jsonb field is no longer decoded by psycopg2 (an empty stub is used instead of json.joads), all decoding stuff is delegated to JSONField.from_db_value(...).
But postgres json field is still decoded by psycopg2, which causes 'TypeError' while trying to decode it again by JSONField.

Breaking commit: https://github.com/django/django/commit/0be51d2226fce030ac9ca840535a524f41e9832c

Example:

class CustomJSONField(models.JSONField):

    def db_type(self, *args, **kwargs):
        # stored as-is, keys order is preserved
        return 'json'

class TestModel(models.Model)
    json_data = CustomJSONField(default=dict)


TestModel(json_data={"foo": "bar"}).save()
TestModel.objects.last()

Expected behavior: code snippet works (as in Django 3.1).

Actual behavior:

TypeError: the JSON object must be str, bytes or bytearray, not dict

The issue was mentioned in few tickets (#31973, #31956) but eventually their authors found workarounds (e.g. by switching to jsonb field) or still using Django <= 3.1. I think support of postgres json (not only jsonb) is important too.

The possible fix is to add the decoder stub for json field too:

Before (https://github.com/django/django/blob/main/django/db/backends/postgresql/base.py#L218):

psycopg2.extras.register_default_jsonb(conn_or_curs=connection, loads=lambda x: x)

After:

psycopg2.extras.register_default_json(conn_or_curs=connection, loads=lambda x: x)
psycopg2.extras.register_default_jsonb(conn_or_curs=connection, loads=lambda x: x)

Change History (1)

comment:1 by Artem, 4 years ago

Description: modified (diff)
Note: See TracTickets for help on using tickets.
Back to Top