Diagram doesn't match text in Middleware documentation

[Chris Petrilli]

In the [current documentation ​http://docs.djangoproject.com/en/dev/topics/http/middleware/], the diagram under Activate Middleware doesn't match the MIDDLEWARE_CLASSES above it.

[Chris Petrilli]

Design with squares instead of circles

[Chris Petrilli]

I have attached a couple designs, and the OmniGraffle document they're derived from. Rather than continue with the circle, I switched to squares in one of them simply because the middleware names were becoming unreadable due to their length.

[Tim Graham]

The attached diagram is out of date, since MessageMiddleware is now included by startproject (#12795). We may want to consider an approach that doesn't require the image to be updated each time this list is changed.

[Chris Petrilli]

I've updated the diagram, but I agree that it could become an issue long-term. Unfortunately, I can't think of a way to express it "visually" in words that will make it clear how the process works.

[Gabriel Hurley]

Patch updates the middleware.png to use the middleware_arrow.png provided on this ticket.

[Gabriel Hurley]

I added a patch that just does a one-for-one replacement of the existing middleware.png file with the middleware-arrow.png provided by petrilli (which is currently up-to-date).

Until a decision is reached about finding a way to avoid using an image at all, we might as well have a working patch for the docs.

[Shai Berger]

Hi,

The images provided, as well as the one currently in the docs, share a problem: They are misleading, because they over-simplify the process.

The points missed:

1. There are actually two passes "going in", one using process_request and one using process_view
2. The "go in, then out" pictures create the impression that process_response will only be called for a given middleware class if the corresponding process_request was called; the text makes it clear that this isn't the case.
3. Exception processing is left completely out of the picture, and that is especially misleading; in particular, it should be made clear that exception processing does not replace response processing, but is done before it; and that it is only applied to the view, not to any processing in the middlewares.

The mismatch of the list of middlewares is, in comparison, completely minor IMHO. Also, I think a simple, complete solution to the list mismatch problem is to use generic names ("MiddleWareA, MiddleWareB" etc.) in the diagram.

[SafPlusPlus]

Diagram of middleware request processing

[SafPlusPlus]

Diagram of middleware view processing

[SafPlusPlus]

Diagram of middleware response processing

[SafPlusPlus]

Diagram of middleware response processing

[SafPlusPlus]

Extension of petrilli's OnmiGraffle file

[SafPlusPlus]

I agree with shai on that the current diagram also doesn't represent correctly how the middleware processing actually works. I took petrilli diagram and extended it with diagrams for the handling of requests, views, exceptions and responses.

Two reasons though why I'm not 100% content with my diagrams. One being that they are ugly and the second being that they may appear too complex to glance over quickly.

[Gabriel Hurley]

I'm gonna unassign this to let somebody else make the final decision during the sprint since I'm mostly not available right now.

[Russell Keith-Magee]

Deferring due to the absence of a trunk-ready patch.

[Shai Berger]

SVG file -- diagram depicting middleware processing

[Shai Berger]

PNG file -- diagram depicting middleware processing

[Shai Berger]

The diagram I've just attached shows, I believe, a true picture of the middleware processing. Black arrows show the normal flow, gray arrows are other possible (non-erroneous) paths. I think the rest is self-explanatory enough.

This is the PNG, for easy viewing; a "source" (SVG) was also attached.

Shai.

[Gabriel Hurley]

This latest diagram is by far the most accurate, but my concern is that it's accurate at the cost of being fairly complicated. I'm not certain there's a better compromise between accuracy and complexity to be had, but I worry this will further confuse inexperienced users.

I think at the very least it should have a key in the diagram explaining what the colors mean (black = normal flow, grey = possible short-circuits, red = exception).

If anyone else can think how this could be further simplified feel free to chime in.

[Aymeric Augustin]

Change UI/UX from NULL to False.

[Aymeric Augustin]

Change Easy pickings from NULL to False.

[Shai Berger]

I would like to point out that with the introduction of process_template_response, the middleware documentation has become unclear and ambiguous: I couldn't fathom, by reading it, if process_template_response calls are all made before process_response calls or instead of them. If the former, than my suggested diagram is no longer correct, and the true picture is even more complex.

[Aymeric Augustin]

We have to strike a balance between completeness and clarity. This graph is displayed at the beginning of the introduction to middleware and should be understandable at first sight. For this reason, I wouldn't attempt to show that each middleware can short-circuit the rest of the chain by returning a response in process_request/view/exception. It's clearly explained in the text and rather intuitive.

However, there's another aspect that I'd like to represent: process_template_response and process_response are applied after process_exception. The current docs describe process_exception at the end — they start with the common case; that can be confusing. They repeat that "Response middleware is always called on every response" but the concrete result is difficult to understand (at least for me) without a visual representation.

I'm attaching (yet another) SVG file that implements these ideas and tries not to be too ugly.

[Aymeric Augustin]

After getting some feedback on IRC, I'm going to commit the second version.

I'll include the OmniGraffle source file in the commit, even though it's in a proprietary format, because we often waste time looking for sources.

[Shai Berger]

I agree that the second version is better, but I still think it doesn't achieve the goals you set for it in comment:16. In particular, it doesn't clarify the relationship between the three middleware functions on the way out.

Here's a quick ascii-art sketch of what I think the diagram should look like. Every block should be expanded to "middleware1, middleware2, middleware3" in the correct order as in my previous suggestion.

Question: is it -- as implied by the suggested diagrams -- that an exception middleware can return a template response to be processed by the template-response middleware? The below represents my guess that this is not the case.

#                       |
#                       V
#              [ process_request ]
#                       |
#                       V
#              [  process_view   ]
#                       |
#                       V
#              [    USER VIEW     ]
#              /        |         \
# [process_exception]   |   [process_template_response]
#              \        V         /
#              [ process_response ]
#                       |
#                       V

(Of course, the arrows to process_exception and process_template_response should be formed and colored to mark their optionality etc).

[Aymeric Augustin <aymeric.augustin@…>]

In ae8e97384bbacaf4094b5f98c83b8599478b3fac:

Fixed #12502 -- Improved middleware diagram.

[Aymeric Augustin]

Replying to shai:

I agree that the second version is better, but I still think it doesn't achieve the goals you set for it in comment:16.

Sorry, I committed before seeing your comment. I wanted to get it done tonight rather than let it linger for another three years :)

Question: is it -- as implied by the suggested diagrams -- that an exception middleware can return a template response to be processed by the template-response middleware?

Yes, this is possible (look at django.core.handlers.base).

There are four possibilities for the response phase:

• exception => template_response => response
• exception => response
• template_response => response
• response

The dashes on the diagram mean that exception and template_response are optional, so these four possibilities are represented.

[Shai Berger]

Oh, Ok, then. I stand corrected. Thanks for finally taking care of this.