Opened 4 years ago

Closed 4 years ago

Last modified 4 years ago

#31819 closed Bug (invalid)

Django + Django REST Framework generates API errors with swagger

Reported by: Perry Harrington Owned by: nobody
Component: Core (Other) Version: 3.0
Severity: Normal Keywords: swagger regex javascrip rest openapi
Cc: Perry Harrington Triage Stage: Unreviewed
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

The Django Validators use the syntax \Z to designate an end of line, similar to the more conventional $ marker.

This swagger ticket gives context to the following: https://github.com/swagger-api/swagger-editor/issues/1601

Javascript uses the $ as the end anchor, this has slightly different semantics to the intended \Z anchor in Python.

The problem is that swagger chokes on this syntax, and looking at the code it comes directly from the Python regexes being exported to the openapi spec.

I propose 2 possible solutions:

  1. Change the Django regexes to use the $ anchor, this may have unintended behavior. The \Z regex is not valid as an anchor in Javascript.
  2. Run all of the python regexes through a simple replace, changing \Z for $

I'm open to any other suggestions, but as I see it now, the yaml produced from Django regexes is not compliant with Javascript and javascript validators by extension.

I can manually edit yaml files, but I'd like to have a dynamic API endpoint for schema definitions.

Change History (6)

comment:1 by Perry Harrington, 4 years ago

Cc: Perry Harrington added

comment:2 by Perry Harrington, 4 years ago

Here's a comparison of the regex APIs for Python and Javascript:

Python:

\Z

Matches only at the end of the string.

$

Matches the end of the string or just before the newline at the end of the string, and in MULTILINE mode also matches before a newline. foo matches both ‘foo’ and ‘foobar’, while the regular expression foo$ matches only ‘foo’. More interestingly, searching for foo.$ in 'foo1\nfoo2\n' matches ‘foo2’ normally, but ‘foo1’ in MULTILINE mode; searching for a single $ in 'foo\n' will find two (empty) matches: one just before the newline, and one at the end of the string.

Javascript:

$

Matches the end of input. If the multiline flag is set to true, also matches immediately before a line break character. For example, /t$/ does not match the "t" in "eater", but does match it in "eat".

PHP:

\Z

end of subject or newline at end (independent of multiline mode)

A dollar character ($) is an assertion which is TRUE only if the current matching point is at the end of the subject string, or immediately before a newline character that is the last character in the string (by default). Dollar ($) need not be the last character of the pattern if a number of alternatives are involved, but it should be the last item in any branch in which it appears. Dollar has no special meaning in a character class.

The meaning of dollar can be changed so that it matches only at the very end of the string, by setting the PCRE_DOLLAR_ENDONLY option at compile or matching time. This does not affect the \Z assertion.

Version 0, edited 4 years ago by Perry Harrington (next)

comment:3 by Carlton Gibson, 4 years ago

Resolution: invalid
Status: newclosed

Hi Perry. This isn't in scope for Django: differences between language implementations of regex isn't something we're going to address.

The particular issue with DRF OpenAPI schema generation should (at least in part) be addressed by this PR, which was merged into DRF. That will be part of DRF 3.12, which is pending, but has been delayed by (let us just call it) 2020, and will be released some time soon. In the meantime you can install the latest master using pip's VCS support.

If you find there are further changes needed, please open a new issue or preferably a PR on the DRF repo: it is appropriate there that we'll do what we can to map between the required formats when generating the OpenAPI schema.

Thanks.

comment:4 by Perry Harrington, 4 years ago

Hi, thank you for your reply.

I looked at the commit and it changes \Z to \z, from my research and reading \z is not a valid anchor in several languages.

I think that $ is a more suitable replacement based on behavior, since \Z matched before a newline like $ in most languages.

Like you said, not Django's problem, DRF's problem.

Last edited 4 years ago by Perry Harrington (previous) (diff)

comment:5 by Perry Harrington, 4 years ago

Quick update because an edit doesn't trigger an email and you won't have context in email:

I see you are the one who merged that pull request, would you care to elaborate on why \z was accepted over $?

For me it's a simple code hack, but I would really like for it to be upstream.

comment:6 by Carlton Gibson, 4 years ago

It's best effort — the contributor suggested that fix (targeting JavaScript I think.)

I'm sceptical that there's any perfect solution here. If you can do better, please do suggest a PR with reasoning on DRF.

Note: See TracTickets for help on using tickets.
Back to Top