Opened 3 years ago

Closed 3 years ago

Last modified 3 years ago

#33584 closed New feature (needsinfo)

Encourage using persistent database connections

Reported by: Tom Carrick Owned by: Tom Carrick
Component: Database layer (models, ORM) Version: dev
Severity: Normal Keywords:
Cc: Florian Apolloner Triage Stage: Unreviewed
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

Sometimes I'm asked to review a Django project's performance. The most typical issues are probably N+1 selects and wasting time creating connections.

I propose encouraging setting CONN_MAX_AGE to a non-zero value by:

  • Setting CONN_MAX_AGE to None or a reasonably high value in the generated settings.py - but keeping the default as-is for compatibility.
  • Adding a deployment level system check to guard against a zero value.

Anyone who needs this to be 0 can easily change it and silence the system check, but I think most projects will benefit from this.

Change History (5)

comment:1 by Tom Carrick, 3 years ago

Owner: changed from nobody to Tom Carrick

comment:2 by Carlton Gibson, 3 years ago

Cc: Florian Apolloner added
Resolution: needsinfo
Status: assignedclosed

Hi Tom. Thanks for the suggestion here.

It looks like this has come up before, and there's a problem with runserver compatibility: mailing list post from 2013.

(Mailing list search for CONN_MAX_AGE)

Do these points still apply? If so, is there a way around them? Did anyone explore the idea from the mailing list thread?

Also relevant is #33497 -- "Database persistent connections do not work with ASGI in 4.0" — Question then: is it clear cut that persistent connections are better — not just for this ASGI case but in general? (If you have lots of traffic, sure, but... 🤷) From that ticket, I know Florian is thinking about a connection pool, which would be connected.

Whilst generally it seems Sure, yeah, I think some wider discussion is needed, even if that just results in Clearly, yeah. (DevelopersMailingList :)
I'll mark this needs info in the meantime.

comment:3 by Tom Carrick, 3 years ago

Thanks, makes sense. I'll do a bit of digging and see where I end up.

in reply to:  2 comment:4 by Florian Apolloner, 3 years ago

Replying to Carlton Gibson:

It looks like this has come up before, and there's a problem with runserver compatibility: mailing list post from 2013.

I think this still holds true. Unless we change runserver to use a threadpool (or db pool) it doesn't make much sense to use persistent connections there.

comment:5 by Tom Carrick, 3 years ago

Okay, I think it's best to revisit this once (if?) we have connection pooling.

Note: See TracTickets for help on using tickets.
Back to Top