Fixes problem in initial_sql where "--" is in a string

[Tim Chase]

In the core/management.py get_custom_sql_for_model() function, if the initial SQL has a "--" in a string, it chokes.

The attached diff should fix the problem. Test cases:

select '--' as foo; -> select '--' as foo;
select '--' as foo; -- select stuff -> select '--' as foo;
select '--' as foo; -- select stuff -- because I want to -> select '--' as foo;
select * foo; -- select stuff -- because I want to -> select * foo;
select * foo; -- select stuff -> select * foo;

[Tim Chase]

"svn diff" against trunk

[shaleh]

updated patch. Seems simpler than the previous one. Added test casesto simple.sql as well as a unittest for custom_sql_for_model itself

[shaleh]

added tests

[Malcolm Tredinnick]

The first patch doesn't work (as demonstrated by the tests in the second patch). I don't like the code changes in the second patch, though. Having to a linear scan through every character, one at a time, in the initial SQL strings seems inefficient. There must be a better way.

[shaleh]

Yeah, I was not entirely happy either BUT how often is this called? Is it really in a time efficient section? In other words, while it may not be 100% pretty do we really care? My understanding is this code only happens at application initialization time from manage.py or similar. We could make a sample sql file that had a thousand or two entries and see just how long it takes.

I am willing to help get this finalized, but I do not see a clear regex only solution out of this.

[jammycakes]

There are a couple of related problems here:

1. It chokes on strings that contain a semicolon
2. It also chokes when you're trying to define stored procedures, e.g.

delimiter $$

create procedure DoSomething()
begin
   select count(*) from MyTable ;
end

$$

delimiter ;

Wouldn't a better solution be to spawn a separate process (mysql -e /path/to/sql/file) to handle the initial SQL as a batch than to try and reinvent the database engine's parser?

[anonymous]

This seems to have been fixed in bug #6394.

[Claude Paroz]

Just a test case demonstrating the issue is still present

[Claude Paroz]

Possible resolution of the issue

[Claude Paroz]

A possible patch, inspired from the initial patch regex to clean comments.

[Claude Paroz <claude@…>]

In [423244bc6b670abc2b7d6896add5c1baf0b4ef2a]:

Fixed #4680 -- Improved initial_sql parsing

In particular, allow the '--' sequence to be present in string
values without being interpreted as comment marker.
Thanks Tim Chase for the report and shaleh for the initial patch.