Oracle unicode insert fails.

[Jani Tiainen]

smart_str (which detection selects) does not work.

using force_unicode works (except for connection string).

Depending on what OCI client 10.2.0.5 or instant client 11.2 is used when compiling cx_Oracle causes variation. 10.2.0.5 doesn't work with smart_str while 11.2 does work.

Both can take plain unicode (u'<some unicode stuff here>') when using just cx_Oracle directly without any problems.

Note:

If I add manually some unicode to database Django can read it without any problems.

[anonymous]

Original story - ​https://groups.google.com/forum/?fromgroups=#!topic/django-users/fXB4wReYth0

[Anssi Kääriäinen]

I haven't haven't been able to verify this (no 10.2.0.5 here). The report seems well researched in the django-users thread so accepted based on that.

[Jani Tiainen]

Problem is that Django makes incorrect detection what to string encoding system to use:

I did also some source code diving to make sure how this works:

In version 5.0.x cx_Oracle.UNICODE is only defined if code was compiled without flag WITH_UNICODE.

In version 5.1+ cx_Oracle.UNICODE is always defined if Python major version compiled against is < 3.

According to changelog of cx_Oracle:

Changes from 5.0.4 to 5.1

1) Remove support for UNICODE mode and permit Unicode to be passed through in

everywhere a string may be passed in. This means that strings will be
passed through to Oracle using the value of the NLS_LANG environment
variable in Python 3.x as well. Doing this eliminated a bunch of problems
that were discovered by using UNICODE mode and also removed an unnecessary
restriction in Python 2.x that Unicode could not be used in connect strings
or SQL statements, for example.

So behaviour should be: If cx_Oracle version is greater or equal to 5.1 real unicode should be used always. With pre-5.1 current behaviour is sufficient.

[Anssi Kääriäinen <akaariai@…>]

In d194f290571f7e9dda7d2fd7a6f2b171120f2f14:

Fixed #19606 -- Adjusted cx_Oracle unicode detection.

[Anssi Kääriäinen <akaariai@…>]

In 457290326fa38297467922d827c630ae2026e88b:

[1.5.x] Fixed #19606 -- Adjusted cx_Oracle unicode detection.

Backpatch of d194f290571f7e9dda7d2fd7a6f2b171120f2f14

[Anssi Kääriäinen]

I felt very uncomfortable about backporting this as I can't actually test the combo this is supposed to fix - that is, cx_oracle 5.1 + instantclient 10.2 as that wont compile on my machine. Based on the patch, cx_oracle 5.0 users should no get any change and I have tested 5.1 + instantclient 11.

I think this falls into the category of patches we can still backpatch to 1.5 at this stage - the combination of cx_oracle 5.1 + instantclient 10.2 is supported, but it doesn't work with unicode values currently.

So, after using too much time trying to tests this, I did backpatch this on the grounds that instanclient 11 works, cx_oracle 5.0 works as always, and the research done by jtiai shows that this is the right thing to do for 5.0 + instantclient 10.

[Jani Tiainen]

According to my research it's all about version of cx_Oracle 5.0.x series does behave have slightly different definition for cx_Oracle.UNICODE than 5.1.x

It has nothing to do with OCI version linked against even I though that was the reason. But I think 1.5 is sufficient at this point.

[Anssi Kääriäinen <akaariai@…>]

In 457290326fa38297467922d827c630ae2026e88b:

[1.5.x] Fixed #19606 -- Adjusted cx_Oracle unicode detection.

Backpatch of d194f290571f7e9dda7d2fd7a6f2b171120f2f14

[anonymous]

#20292 is possibly related, at least reverting this change fixes that bug.