Opened 3 years ago

Closed 3 years ago

#33161 closed New feature (fixed)

Do not ignore transaction durability errors within TestCase

Reported by: Krzysztof Jagiełło Owned by: Krzysztof Jagiełło
Component: Testing framework Version: dev
Severity: Normal Keywords:
Cc: David Seddon, Adam Johnson, Ian Foote, Hannes Ljungberg Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description (last modified by Krzysztof Jagiełło)

Currently there is a discrepancy in how durable atomic blocks are handled in TransactionTestCase vs TestCase. Using the former, nested durable atomic blocks will, as expected, result in a RuntimeError. Using the latter however, the error will go unnoticed as the durability check is turned off.

I have faced some issues with this behaviour in a codebase where we heavily utilize TestCase and where we recently started to introduce durable atomic blocks – the durability errors do not surface until the code hits staging/production. The solution could be to switch over to TransactionTestCase for the test classes that hit code paths with durable atomic blocks, but having to identify which tests could be affected by this issue is a bit inconvenient. And then there is the performance penalty of using TransactionTestCase.

So, to the issue at hand. The durability check is disabled for TestCase because otherwise durable atomic blocks would fail immediately as TestCase wraps its tests in transactions. We could however add a marker to the transactions created by TestCase, keep a stack of active transactions and make the durability check take the stack of transactions with their respective markers into account. This way we could easily detect when a durable atomic block is directly within a transaction created by TestCase and skip the durability check only for this specific scenario.

To better illustrate what I am proposing here, I have prepared a PoC patch. Let me know what you think!

Patch: https://github.com/django/django/pull/14919

Change History (18)

comment:1 by Krzysztof Jagiełło, 3 years ago

Description: modified (diff)

comment:2 by Krzysztof Jagiełło, 3 years ago

Owner: changed from nobody to Krzysztof Jagiełło
Status: newassigned

comment:3 by Krzysztof Jagiełło, 3 years ago

Version: 3.2dev

comment:4 by Mariusz Felisiak, 3 years ago

Cc: David Seddon Adam Johnson Ian Foote added

I don't think it's worth additional complexity and potential slowdown of TestCase for all users. I will wait for a second opinion before making a decision.

comment:5 by Krzysztof Jagiełło, 3 years ago

Could you elaborate on what would cause the slowdown? I have noticed the mention about it in the docs, but couldn't really figure out why this would incur any performance penalty.

Version 0, edited 3 years ago by Krzysztof Jagiełło (next)

comment:6 by Hannes Ljungberg, 3 years ago

Cc: Hannes Ljungberg added

comment:7 by Adam Johnson, 3 years ago

I think it's reasonable for tests to still detect such problematic durable atomic blocks, without requiring TransactionTestCase.

I've reviewed the initial PR and I came up with an idea for a simpler implementation: https://github.com/django/django/pull/14922

My implementation removes ensure_durability and adds another boolean flag on BaseDatabaseWrapper. This leaves us with about the same level of complexity as before the PR, so hopefully that alleviates any concerns.

(I also don't believe there's any potential for notable slowdown from the initial implementation - it only adds one stack.)

comment:8 by Mariusz Felisiak, 3 years ago

Triage Stage: UnreviewedAccepted

(I also don't believe there's any potential for notable slowdown from the initial implementation - it only adds one stack.)

Persuaded.

comment:9 by Krzysztof Jagiełło, 3 years ago

Description: modified (diff)

comment:10 by Krzysztof Jagiełło, 3 years ago

Thanks Adam! Your approach seems more straightforward to me than the initial implementation. Keeping around the stack of atomic blocks just to handle this is specific scenario was a bit unnecessary.

comment:11 by Krzysztof Jagiełło, 3 years ago

Description: modified (diff)

comment:12 by Krzysztof Jagiełło, 3 years ago

It took me a while to realize that a boolean flag on BaseDatabaseWrapper might not be enough for cases like these:

    def test_sequence_of_durables(self):
        with transaction.atomic(durable=True):
            ...
        with transaction.atomic(durable=True):
            ...

Keeping the atomic block stack around makes it easier to handle it correctly.

comment:13 by Krzysztof Jagiełło, 3 years ago

Description: modified (diff)

comment:14 by David Seddon, 3 years ago

Thanks for looking into this! I think it's a great idea.

It reminds me of the difficulties of testing code that uses select_for_update. I have seen many occasions where wrapped test cases are the only coverage of such code, and we don't see the issue until it hits production.

The approach proposed here should make it less likely that unrunnable code will make its way onto production.

[Edited from my original post now I better understand the behaviour.]

Last edited 3 years ago by David Seddon (previous) (diff)

comment:15 by Krzysztof Jagiełło, 3 years ago

Replying to David Seddon:

It reminds me of the difficulties of testing code that uses select_for_update. I have seen many occasions where wrapped test cases are the only coverage of such code, and we don't see the issue until it hits production.

Got hit by that one to in the past too! I think that once this patch gets merged we should be able to apply a similar fix to select_for_update too.

Regarding the patch, I have decided to keep working on my PR (https://github.com/django/django/pull/14919) as the one proposed by Adam did not work for sequential durable transactions (see the repro case in my previous comment). Let me know if there is anything you would like me to adjust in my PR before its merge-ready.

comment:16 by Krzysztof Jagiełło, 3 years ago

Description: modified (diff)

comment:17 by Mariusz Felisiak, 3 years ago

Triage Stage: AcceptedReady for checkin

comment:18 by Mariusz Felisiak <felisiak.mariusz@…>, 3 years ago

Resolution: fixed
Status: assignedclosed

In 8d9827c0:

Fixed #33161 -- Enabled durability check for nested atomic blocks in TestCase.

Co-Authored-By: Adam Johnson <me@…>

Note: See TracTickets for help on using tickets.
Back to Top