Management commands migration to argparse

[José Luis Patiño Andrés]

The class BaseCommand, from which developers subclass their own management commands, is using the ​`optparse` module, which is actually deprecated.

If there are no important reasons to remain using the deprecated module, perhaps would be a good idea to port the BaseCommand code to the new module ​`argparse`, which will be used in Python 3.

[Ramiro Morales]

Good idea, has been discussed before (but there isn't anassociated ticket so thanks for opening this one).

But possible only once the minimal Python version supported by Django gets to the 2.7 level, such is the version where argparse got added to the stdlib.

[José Luis Patiño Andrés]

Ok. I think that since I've took part some days ago in fixing the ticket #19877 and I've got familiar with that part, I can also try to fix this one. So it can be appended to Django master code when it reaches the 2.7 Python as minimal version supported.

[Carl Meyer]

We'll need to provide a deprecation path and migration strategy for authors of custom management commands, since optparse is exposed directly to them via the definition of additional options for a custom command.

[Marc Tamlyn]

We are now able to do this as Django master is 2.7 only. I'm not having many great ideas about how to support both at once, my instinct says we'll need to do something like providing a flag on the Command instance to say "use_argpase" instead. Even this isn't clear in terms of how it would work.

Then again, there is no planned removal date for optparse (​http://www.python.org/dev/peps/pep-0389/#deprecation-of-optparse) so it's not really urgent.

[Claude Paroz]

Before starting with the implementation, there is a design decision to make. argparse doesn't provide the same declarative syntax that optparse used to provide with make_option. I see two main ways to go forward:

1. Keep declarative syntax and mostly the same Command API that we have now. Instead of importing make_option from optparse, we would import it from a Django-provided utility, optionally renaming it make_argument. Internally, this would only build a list of parameters that we would then pass to add_argument calls in create_parser.
   Advantages: nicer declarative syntax, less changes for custom command writers (and accessorily easier for call_command to get default values without instantiating the parser)

2. Depart from the current declarative syntax and change the documentation about adding options for custom commands, basically instructing to subclass create_parser (or more probably create a new subclassable method BaseCommand.add_arguments(self, parser)) and to add arguments by calling parser.add_argument(<params>).
   Advantages: closer approach to standard Python

[Jerome Leclanche]

So on Pootle we've been working on moving away from optparse in all our backend scripts but due to the dependency django has on optparse for specifically this, we are forced to use both optparse and argparse in the code which makes kittens cry.

@claudep I would go with option 2. The advantage here is that it is much more powerful right out of the box. Subcommands would create their own parser and would be able to inherit the parent parser simply by calling super(). This allows easy creation of things such as subcommands and such.

Being declarative is nice in principle but doesn't really work out for argument parsing - there is a reason python changed the logic there.

Are you planning to work on this bug? I would love to help, FWIW.

[Claude Paroz]

Thanks Adys for your input. Feel free to start hacking on it, and share your work.

[Claude Paroz]

@Adys, I made some progress on this issue. If you have also worked on this, we might share our work, otherwise just wait that my branch is in a publishable state...

[Claude Paroz <claude@…>]

In 96e4b52ab2c6fae3e46a197d44c42cb6ebde7d9c:

Converted Django scripts to argparse

Refs #19973.

[Claude Paroz]

Here is the pull request with my work so far: ​https://github.com/django/django/pull/2780

Comments welcome.

[Tim Graham]

Claude, do you know if this will obsolete #11407?

[Claude Paroz]

Hopefully, yes, but in any case, I'll revisit #11407 after committing this one.

[Claude Paroz]

Fixed in:

• 856863860352aa1f0288e6c9168a0e423c4f5184 Fixed #19973 -- Replaced optparse by argparse in management commands
• cbff097bd91fad42c7231026968f686598b1d7a2 Documented optparse to argparse changes for management commands
• 4b4524291adbc78ab880317124803fc37a2e414a Converted test management command to argparse
• f17b24e407385eb18651bf023a187347aa9c1f75 Converted remaining management commands to argparse

[Claude Paroz <claude@…>]

In 5949c2118d7cc7cffe86cd9db1820b186b88bfc0:

Restored command error behavior when called from command line

Refs #19973.

[Tim Graham <timograham@…>]

In 6a70cb53:

Refs #19973 -- Removed optparse support in management commands per deprecation timeline.

[Tim Graham <timograham@…>]

In ccd5a23f:

Fixed #27000 -- Removed BaseCommand.usage() per deprecation timeline (refs #19973).

[Tim Graham <timograham@…>]

In 7dd8e53c:

[1.10.x] Fixed #27000 -- Removed BaseCommand.usage() per deprecation timeline (refs #19973).

Backport of ccd5a23fba91e66656bb19b60bb6d737cab728f5 from master