Changes between Version 4 and Version 5 of Ticket #36233, comment 7
- Timestamp:
- Mar 9, 2025, 4:56:40 AM (13 hours ago)
Legend:
- Unmodified
- Added
- Removed
- Modified
-
Ticket #36233, comment 7
v4 v5 39 39 ---- 40 40 41 As we can see, the context variable passed to quantize() expects the result of the quantization to have a precision of 16, however, if the input is 9999999999999999 (16 digits), the actual number of significant digits after the quantize operation becomes 18 (9999999999999999.00). This will raise an InvalidOperation exception, see - [https://docs.python.org/3/library/decimal.html#decimal.Decimal.quantize] 41 When decimal_places is 0 and the input is 9999999999999999 (16 digits) the result of the quantization is 10000000000000000 (17 digits), this will raise an InvalidOperation exception, see - [https://docs.python.org/3/library/decimal.html#decimal.Decimal.quantize]. I assume this rounding occurs because of SQLite's quirk of only preserving the first 15 significant bits. 42 42 43 In the scenario where decimal_places is 0 and the input is 9999999999999999 (16 digits) the result of the quantization is 10000000000000000 (17 digits), i assume this is because of SQLite's quirk of only preserving the first 15 significant bits. This is a separate issue albeit the error is the same. 44 45 ---- 46 47 I hope the above made sense. I'm not sure what approach I should follow to fix this issue, any help would be appreciated. 43 I hope the above made sense. I'm not sure what approach I should follow to fix this issue, does it make sense to increase the precision? Any help would be appreciated.