Django + Django REST Framework generates API errors with swagger

[Perry Harrington]

The Django Validators use the syntax \Z to designate an end of line, similar to the more conventional $ marker.

This swagger ticket gives context to the following: ​https://github.com/swagger-api/swagger-editor/issues/1601

Javascript uses the $ as the end anchor, this has slightly different semantics to the intended \Z anchor in Python.

The problem is that swagger chokes on this syntax, and looking at the code it comes directly from the Python regexes being exported to the openapi spec.

I propose 2 possible solutions:

1. Change the Django regexes to use the $ anchor, this may have unintended behavior. The \Z regex is not valid as an anchor in Javascript.
2. Run all of the python regexes through a simple replace, changing \Z for $

I'm open to any other suggestions, but as I see it now, the yaml produced from Django regexes is not compliant with Javascript and javascript validators by extension.

I can manually edit yaml files, but I'd like to have a dynamic API endpoint for schema definitions.

[Perry Harrington]

Here's a comparison of the regex APIs for Python and Javascript:

Python:

\Z

Matches only at the end of the string.

$

Matches the end of the string or just before the newline at the end of the string, and in MULTILINE mode also matches before a newline. foo matches both ‘foo’ and ‘foobar’, while the regular expression foo$ matches only ‘foo’. More interestingly, searching for foo.$ in 'foo1\nfoo2\n' matches ‘foo2’ normally, but ‘foo1’ in MULTILINE mode; searching for a single $ in 'foo\n' will find two (empty) matches: one just before the newline, and one at the end of the string.

Javascript:

$

Matches the end of input. If the multiline flag is set to true, also matches immediately before a line break character. For example, /t$/ does not match the "t" in "eater", but does match it in "eat".

PHP:

\Z

end of subject or newline at end (independent of multiline mode)

$

A dollar character ($) is an assertion which is TRUE only if the current matching point is at the end of the subject string, or immediately before a newline character that is the last character in the string (by default). Dollar ($) need not be the last character of the pattern if a number of alternatives are involved, but it should be the last item in any branch in which it appears. Dollar has no special meaning in a character class.
The meaning of dollar can be changed so that it matches only at the very end of the string, by setting the PCRE_DOLLAR_ENDONLY option at compile or matching time. This does not affect the \Z assertion.

[Carlton Gibson]

Hi Perry. This isn't in scope for Django: differences between language implementations of regex isn't something we're going to address.

The particular issue with DRF OpenAPI schema generation should (at least in part) be addressed by ​this PR, which was merged into DRF. That will be part of DRF 3.12, which is pending, but has been delayed by (let us just call it) 2020, and will be released some time soon. In the meantime you can install the latest master using ​pip's VCS support.

If you find there are further changes needed, please open a new issue or preferably a PR on the DRF repo: it is appropriate there that we'll do what we can to map between the required formats when generating the OpenAPI schema.

Thanks.

[Perry Harrington]

Hi, thank you for your reply.

I looked at the commit and it changes \Z to \z, from my research and reading \z is not a valid anchor in several languages.

I think that $ is a more suitable replacement based on behavior, since \Z matched before a newline like $ in most languages.

Like you said, not Django's problem, DRF's problem.

Let's see if I get any traction on that front.

I see you are the one who merged that pull request, would you care to elaborate on why \z was accepted over $?

For me it's a simple code hack, but I would really like for it to be upstream.

[Perry Harrington]

Quick update because an edit doesn't trigger an email and you won't have context in email:

I see you are the one who merged that pull request, would you care to elaborate on why \z was accepted over $?

For me it's a simple code hack, but I would really like for it to be upstream.

[Carlton Gibson]

It's best effort — the contributor suggested that fix (targeting JavaScript I think.)

I'm sceptical that there's any perfect solution here. If you can do better, please do suggest a PR with reasoning on DRF.