Improve connection._dirty flag handling

[Anssi Kääriäinen]

The connection._dirty flag currently represents a mix of things:

• It is sometimes used for tracking if transaction management is enabled (enabled when self._dirty is not None). This doesn't work correctly, as the _dirty flag is never reset to None. And, three valued flag isn't a good idea anyways.
• It is used to track if the connection needs commit, but only inside transaction management. When outside tx management it will not be set by Django.

Patch available here: ​https://github.com/akaariai/django/commit/cd31b74c01eca1744bc07f1b462ce73a78666fa6. I believe the commit resolves a bunch of subtle bugs, but it likely also causes a bunch of subtle backwards incompatibilities. One such change is that .commit() and .rollback() can now be called always, not only inside transaction management.

I tried the patch and it passes all tests on sqlite and postgresql (with autocommit off, autocommit on I had time to run just backends transactions transactions_regress and these pass).

With postgresql I also ran the tests with transaction state tracking enabled (the commented out code in psycopg2/base.py). There is one false positive (Django believes the transaction to be in transaction, in reality it isn't) caused by test_cursor_executemany_with_empty_params_list in backends tests. executemany with empty params does nothing, but the dirty flag handling doesn't spot this. This is likely a non issue (or, if this is a issue, it is a bug in current Django, too). There are no false negatives (Django doesn't believe the connection to be in tx, but it is).

The patch also contains a bunch of fixes to tests doing various weird things with transaction management.

[Anssi Kääriäinen]

A new version of the patch available from ​​https://github.com/akaariai/django/commit/50328f0a618674b7143d86acaa7016c5293e9774

I think this one is ready for commit now.

[Anssi Kääriäinen <akaariai@…>]

In 50328f0a618674b7143d86acaa7016c5293e9774:

Fixed #19861 -- Transaction ._dirty flag improvement

There were a couple of errors in ._dirty flag handling:

• It started as None, but was never reset to None.
• The _dirty flag was sometimes used to indicate if the connection was inside transaction management, but this was not done consistently. This also meant the flag had three separate values.
• The None value had a special meaning, causing for example inability to commit() on new connection unless enter/leave tx management was done.
• The _dirty was tracking "connection in transaction" state, but only in managed transactions.
• Some tests never reset the transaction state of the used connection.
• And some additional less important changes.

This commit has some potential for regressions, but as the above list
shows, the current situation isn't perfect either.