Opened 6 years ago

Closed 6 years ago

#29973 closed New feature (fixed)

Add compilemessages --ignore option for excluding search paths

Reported by: Elger Jonker Owned by: Ryan Siemens
Component: Core (Management commands) Version: 2.1
Severity: Normal Keywords: compilemessages, ignore, translation, internationalization
Cc: Ryan Siemens, Simon Charette Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

The makemessages command contain an ignore flag, this allows you to skip directories like 'vendor'.

In our project we have a directory called .tox, this directory is ignored during makemessages. Yet, when compilemessages is run, the .tox directory (which contains a lot of sources) is included with compiling translations.

Now we might have things wrong, but it seems strange to me that makemessages has ignore options, while compilemessages doesn't. Even if it's wrong, we might still want to ignore this folder because we're clumsy and it works just 'good enough'.

The command we'd like to use looks like:

django compilemessages --ignore vendor --ignore .tox -l EN

Change History (17)

comment:1 by Tim Graham, 6 years ago

Resolution: invalid
Status: newclosed

I guess you want to use the --exclude option.

$ python manage.py compilemessages --ignore vendor
usage: manage.py compilemessages [-h] [--locale LOCALE] [--exclude EXCLUDE]
                                 [--use-fuzzy] [--version] [-v {0,1,2,3}]
                                 [--settings SETTINGS]
                                 [--pythonpath PYTHONPATH] [--traceback]
                                 [--no-color] [--force-color]
manage.py compilemessages: error: unrecognized arguments: --ignore vendor

See TicketClosingReasons/UseSupportChannels for places to get help before filing a bug report.

comment:2 by Elger Jonker, 6 years ago

Resolution: invalid
Status: closednew

The --exclude command is about locales. This issue is about having too many directories being used in compilemessages.

Here is an example where i'm in need of an --ignore during compilemessages, including the line where the issue resides and an example output of where things go wrong:

Using this command:

django-admin compilemessages -l NL --exclude DE

And this filter on what directory constitutes a 'locale directory':
https://github.com/django/django/blob/eac9ab7ebb1ce0cbbc79c4cf65e8f70b0635a240/django/core/management/commands/compilemessages.py#L69

It's clear that the filter is far too greedy. And in my case it results in a list of directories that should not be included. Hence the request for an --ignore option just like in makemessages.

The directory filter currently delivers this output on my machine, again: everything named 'locale' is included.

['conf/locale',
 'locale',
 'locale',
 './locale',
 './.tox/default/lib/python3.6/site-packages/import_export/locale',
 './.tox/default/lib/python3.6/site-packages/jet/locale',
 './.tox/default/lib/python3.6/site-packages/jet/dashboard/locale',
 './.tox/default/lib/python3.6/site-packages/constance/locale',
 './.tox/default/lib/python3.6/site-packages/adminsortable2/locale',
 './.tox/default/lib/python3.6/site-packages/leaflet/locale',
 './.tox/default/lib/python3.6/site-packages/dal_select2/locale',
 './.tox/default/lib/python3.6/site-packages/sphinx/locale',
 './.tox/default/lib/python3.6/site-packages/django_countries/locale',
 './.tox/default/lib/python3.6/site-packages/django_extensions/locale',
 './.tox/default/lib/python3.6/site-packages/django/contrib/auth/locale',
 './.tox/default/lib/python3.6/site-packages/django/contrib/admin/locale',
 './.tox/default/lib/python3.6/site-packages/django/contrib/flatpages/locale',
 './.tox/default/lib/python3.6/site-packages/django/contrib/sites/locale',
 './.tox/default/lib/python3.6/site-packages/django/contrib/postgres/locale',
 './.tox/default/lib/python3.6/site-packages/django/contrib/redirects/locale',
 './.tox/default/lib/python3.6/site-packages/django/contrib/sessions/locale',
 './.tox/default/lib/python3.6/site-packages/django/contrib/humanize/locale',
 './.tox/default/lib/python3.6/site-packages/django/contrib/contenttypes/locale',
 './.tox/default/lib/python3.6/site-packages/django/contrib/gis/locale',
 './.tox/default/lib/python3.6/site-packages/django/contrib/admindocs/locale',
 './.tox/default/lib/python3.6/site-packages/django/conf/locale',
 './.tox/default/lib/python3.6/site-packages/debug_toolbar/locale',
 './failmap/locale',
 './failmap/organizations/locale',
 './failmap/game/locale',
 './failmap/map/locale',
 './vendor/dnscheck/engine/locale']

You can also run it for yourself, by creating a 'locale' directory somewhere and then run this part of the django source code:
https://github.com/django/django/blob/eac9ab7ebb1ce0cbbc79c4cf65e8f70b0635a240/django/core/management/commands/compilemessages.py#L67

basedirs=[]
for dirpath, dirnames, filenames in os.walk('.', topdown=True): 
    for dirname in dirnames: 
        if dirname == 'locale': 
            basedirs.append(os.path.join(dirpath, dirname)) 

basedirs

comment:3 by Manu Pascual, 6 years ago

Owner: changed from nobody to Manu Pascual
Status: newassigned

comment:4 by Ryan Siemens, 6 years ago

Hey Manu,

I have a patch in place for this. Do you mind if I take over?

comment:5 by Manu Pascual, 6 years ago

Of course, I would have liked to do it this last weekend, but I was sick. So all your sir.

comment:6 by Manu Pascual, 6 years ago

Owner: Manu Pascual removed
Status: assignednew

comment:7 by Ryan Siemens, 6 years ago

Owner: set to Ryan Siemens
Status: newassigned

comment:8 by Ryan Siemens, 6 years ago

Cc: Ryan Siemens added
Has patch: set

comment:9 by Ryan Siemens, 6 years ago

Type: Cleanup/optimizationNew feature

comment:10 by Ryan Siemens, 6 years ago

I had initially implemented this to just drop a directory by name, but I think thats actually incorrect.
Consider a directory structure like so

./
 |- locale/
 |- foo/
   |- locale/
 |- bar/
   |- foo/
     |- locale/

With my current patch doing compilemessages --ignore=foo would drop ./foo and ./bar/foo from being searched, but I think it should only drop dirs relative to the path. So in this case only ./foo wouldn't be searched. Also I think I it should support globbing similar to makemessages --ignore. Thoughts or opinions?

Thanks.

Last edited 6 years ago by Ryan Siemens (previous) (diff)

comment:11 by Carlton Gibson, 6 years ago

Triage Stage: UnreviewedAccepted

OK, yes. This seems a reasonable addition to mirror makemessages.

comment:12 by Simon Charette, 6 years ago

Cc: Simon Charette added
Patch needs improvement: set

Left a few comments for non-critical improvements.

comment:13 by Ryan Siemens, 6 years ago

Patch needs improvement: unset

Resolved PR review comments. Thanks for the thorough review Simon!

Last edited 6 years ago by Ryan Siemens (previous) (diff)

comment:14 by Simon Charette, 6 years ago

Easy pickings: unset
Triage Stage: AcceptedReady for checkin

Except for the conflict in the 2.2 release notes that'll to be either fixed or changed to target 3.0 the patch looks RFC to me.

comment:15 by Tim Graham, 6 years ago

Summary: compilemessages misses ignore option, compiles more than neededAdd compilemessages --ignore option for excluding search paths

comment:16 by Tim Graham <timograham@…>, 6 years ago

In bc9f0b3:

Refs #29973 -- Extracted helper functions from makemessages.

comment:17 by Tim Graham <timograham@…>, 6 years ago

Resolution: fixed
Status: assignedclosed

In a168e56:

Fixed #29973 -- Added compilemessages --ignore option.

Note: See TracTickets for help on using tickets.
Back to Top