Opened 5 weeks ago
Last modified 5 weeks ago
#35972 assigned Bug
Custom lookup example raises TypeError when looked up against a Subquery
Reported by: | Jacob Walls | Owned by: | Jacob Walls |
---|---|---|---|
Component: | Database layer (models, ORM) | Version: | dev |
Severity: | Normal | Keywords: | regex, mysql |
Cc: | Triage Stage: | Accepted | |
Has patch: | no | Needs documentation: | no |
Needs tests: | no | Patch needs improvement: | no |
Easy pickings: | no | UI/UX: | no |
Description
On the forum I shared the TypeError I was getting when registering a custom lookup for a JSONField
:
File models/lookups.py:13, in NotEqual.as_sql(self, compiler, connection) 11 lhs, lhs_params = self.process_lhs(compiler, connection) 12 rhs, rhs_params = self.process_rhs(compiler, connection) ---> 13 params = lhs_params + rhs_params 14 return "%s <> %s" % (lhs, rhs), params TypeError: can only concatenate list (not "tuple") to list
Since this problematic pattern is documented, we should either fix the documentation or fix the underlying reason it doesn't work. I haven't looked into whether there are backwards-compatible ways to do the latter.
Reproduction is just to follow the documented pattern and register it with a JSONField e.g. @JSONField.register_lookup
, and then try to use it in an ORM query.
Change History (5)
comment:1 by , 5 weeks ago
Resolution: | → worksforme |
---|---|
Status: | new → closed |
Summary: | Custom lookup example raises TypeError → Custom lookup example raises TypeError when used on a JSONField |
comment:2 by , 5 weeks ago
Resolution: | worksforme |
---|---|
Status: | closed → new |
Summary: | Custom lookup example raises TypeError when used on a JSONField → Custom lookup example raises TypeError when looked up against a Subquery |
Thanks Sarah for stubbing out a test. Sorry I didn't notice the special ingredient was Subquery and not JSONField, although I take it there could be other offenders besides Subquery?
This fails:
-
tests/custom_lookups/tests.py
diff --git a/tests/custom_lookups/tests.py b/tests/custom_lookups/tests.py index 2f4ea0a9a0..b1681621c6 100644
a b class LookupTests(TestCase): 249 249 self.assertSequenceEqual(qs1, [a1]) 250 250 self.assertSequenceEqual(qs2, [a1]) 251 251 252 def test_custom_lookup_with_subquery(self): 253 class NotEqual(models.Lookup): 254 lookup_name = "ne" 255 256 def as_sql(self, compiler, connection): 257 lhs, lhs_params = self.process_lhs(compiler, connection) 258 rhs, rhs_params = self.process_rhs(compiler, connection) 259 params = lhs_params + rhs_params 260 return "%s <> %s" % (lhs, rhs), params 261 262 author = Author.objects.create(name="Isabella") 263 264 with register_lookup(models.Field, NotEqual): 265 qs = Author.objects.annotate( 266 unknown_age=models.Subquery( 267 Author.objects.filter(age__isnull=True).values("name") 268 ) 269 ).filter(unknown_age__ne="Plato") 270 self.assertSequenceEqual(qs, [author]) 271 252 272 def test_custom_exact_lookup_none_rhs(self): 253 273 """ 254 274 __exact=None is transformed to __isnull=True if a custom lookup class
comment:4 by , 5 weeks ago
Component: | Documentation → Database layer (models, ORM) |
---|---|
Keywords: | regex mysql added |
I originally framed it as a documentation issue, given that a cleanup/optimization to harden this is probably blocked on a DEP to type-annotate the ORM, but here is a test that fails on MariaDB using only built in lookups (almost certainly on MySQL as well), so we do have a bug in core:
-
tests/custom_lookups/tests.py
diff --git a/tests/custom_lookups/tests.py b/tests/custom_lookups/tests.py index 2f4ea0a9a0..0fe21eb48c 100644
a b class LookupTests(TestCase): 249 249 self.assertSequenceEqual(qs1, [a1]) 250 250 self.assertSequenceEqual(qs2, [a1]) 251 251 252 def test_regex_lookup_with_subquery(self): 253 author = Author.objects.create(name="Isabella") 254 255 qs = Author.objects.annotate( 256 unknown_age=models.Subquery( 257 Author.objects.filter(age__isnull=True).values("name") 258 ) 259 ).filter(name__regex=models.F("unknown_age")) 260 self.assertSequenceEqual(qs, [author]) 261 252 262 def test_custom_exact_lookup_none_rhs(self): 253 263 """ 254 264 __exact=None is transformed to __isnull=True if a custom lookup class
So I think a documentation update is still worthwhile, but after that we should leave this open until we fix the regex lookup or do that DEP :D
comment:5 by , 5 weeks ago
Owner: | set to |
---|---|
Status: | new → assigned |
Testing against main with postgres 17 I don't get the TypeError but I get
django.db.utils.DataError: invalid input syntax for type json
(but I think that's expected as the documented example is not for JSON fields)So far I found no issues with the documented example
This is what I have:
tests/custom_lookups/models.py
tests/custom_lookups/tests.py