[patch] sqlinitialdata is ignoring semi-colons

[ian@…]

currently it seems to be stripping of the semi-colons at the end of a SQL statement.. which means mysql can't import.

Index: management.py
===================================================================
--- management.py       (revision 3007)
+++ management.py       (working copy)
@@ -336,7 +336,7 @@
             (?:[^;'"]+) # not the end of a statement or start of a quote
           | (?:'[^']*') # something in single quotes
           | (?:"[^"]*") # something in double quotes
-        )+)""", re.VERBOSE)
+        )+;?)""", re.VERBOSE)
 
     # Find custom SQL, if it's available.
     sql_files = [os.path.join(app_dir, "%s.%s.sql" % (opts.object_name.lower(), settings.DATABASE_ENGINE)),

[jpellerin@…]

See also #2034 -- I think both patches are needed to handle all cases. (Though I think this one should be modified to make the trailing semi-colon optional -- it should be legal to have an initial data file containing a single statement with no semi-colon.)

On the other hand, it might be better to add a flag to each backend that indicates whether it can accept multiple statements in one execute(), and only try to split the initial data files up for those backends that can't. The ones that can are always going to do a better job of parsing a big ball of sql into statements than any regex.

[ian@…]

hi Jpellerin.

I thought the ';?' meant the semi-colon was optional.
anyway.. what your saying wouldn't work as the file get's catted together with other ones, and
has a 'BEGIN;' and 'COMMIT;' added to it so the semi-colon can't be optional.

does postgres/other backends like ';' to differentiate commands?

[jpellerin@…]

Ian,

You're quite right about the ;?... I should learn not to try to read regexes while half asleep. Sorry.

However, the sql files aren't catted together to send to the backend -- otherwise my original patch to split the initial data file into statements wouldn't do anything. Each file (or, after the patch, each statement) is appended to a list, and each item in the list is sent to the backend as a single query, so a file with one statement and no trailing ; works fine.

[Joeboy]

But if you're doing, say, an sqlreset and piping it to mysql then surely the lines will be concatenated.

[jpellerin@…]

No, in that case also, the collected chunks of sql are executed one at a time. Take a look: source:django/trunk/django/core/management.py#L585. They look catted if you run the commands that print the sql to the console, but when django itself executes the sql, it iterates over a list of statements.

[Joeboy]

Sure, that's true, when django itself is executing the sql. I notice that there seems to be a new 'reset' command that executes the sqlreset code, but even so shouldn't piping it continue to be supported? It looks like manage.py still has operations that require you to send the sql to the db yourself.

Is it a big deal to make sure there's a semicolon at the end of the line?

[Malcolm Tredinnick]

This was fixed in [3178].

[Adrian Holovaty]

Milestone Version 0.92 deleted