Change default transaction isolation level to READ COMMITTED on MySQL

[Shai Berger]

There have been plenty of bug reports and discussions about the problems caused for users by MySQL's implementation of the REPEATABLE READ transaction isolation level, and the fact that it is the default. Django is mostly written for READ COMMITTED; of all supported backends, MySQL is the only one to use a different level by default. Things will probably run more in line with users expectations under READ COMMITTED, and reusable apps' behavior will be more similar across database backends.

Some history: This has been raised already in the 1.2 era, and possibly even before that; the question was raised in different forms again and again until one more instance caused it to be brought up on the mailing list. To be sure, it wasn't the first time the issue was brought to the list either, but ​this discussion seemed to conclude with a rough consensus that the default transaction isolation level for MySQL should change. As far as I'm aware, the issue has not been discussed since then.

There are two kinds of backwards-compatibility problems I already see with this change:

1) The obvious one -- Apps that are written to be correct under MySQL's REPEATABLE READ isolation level may become subtly incorrect by default. This is a very real problem, and hard to tackle because writing tests to capture the problems is very hard -- one needs to make and use at least two separate connections to the same database, and the Django ORM (and testing framework) does not make that easy. The counter-argument to that is that we likely have plenty of apps -- reusable or otherwise -- that are currently subtly wrong because MySQL's REPEATABLE READ semantics is surprising and unintuitive.

2) The less obvious one -- the way to set transaction isolation levels is with SQL executed when the connection is opened, and we need to make sure we don't interrupt with the user's own init_command if there is one.

Some very basic intro to transaction isolation levels in general and MySQL's REPEATABLE READ in particular is in my DC.EU.2016 lightning talk about this:
​https://opbeat.com/community/posts/lightning-talks-day-1/ (starting around 4:00).

[Adam Johnson]

An alternative to changing the default would be to have a system check highly recommending READ-COMMITTED, similar to what we do for MySQL's sql_mode (see ​https://github.com/django/django/blob/master/django/db/backends/mysql/validation.py#L6 ). This would also help educate users a bit more about the change. Possibly this could be part of a path to changing the default... "Warning: In Django 2.1 the default tx_isolation will change..."

[Shai Berger]

Replying to Adam Chainz:

An alternative to changing the default would be to have a system check highly recommending READ-COMMITTED [...] "Warning: In Django 2.1 the default tx_isolation will change..."

That may indeed be the way to go. Two issues arise:

1) We need to make sure which deprecations can happen when (WRT our deprecation policies around LTSs)
2) It may be seen as annoying: "if you know that it's correct to use that other level, why don't you just go and do it? why bug me?"

The big advantage is that it completely solves my 2nd backward-compatibility issue.

[Karen Tracey]

We really should do this! (Unfortunately I have no energy/time to devote. But I did fix up the ticket links in the description.)