AutoSlugField, that can recreate unique slugs during saving.

[Jari Pennanen]

Hi!

I would like to see something that can create unique slugs during saving of the model like ​AutoSlugField by GaretJax in djangosnippets. That specific implementation might be too simple and not enough configurable, but if it were improved a little perhaps? There are several slug hacks in djangosnippets alone that tries to implement unique slugs, with varying results.

Naturally it should be part of Django since it is used by almost every project and every model. Housing fleet of utility libraries is not the nicest way to implement programs, I have stumbled with that a lot in the past.

[Jari Pennanen]

I just ​remixed some code and created a ''new'' AutoSlugField, which too must be improved but it works for me now.

[Jari Pennanen]

There is also one in ​django-extensions but these utility libraries has been weak in past as the maintainers tend to loose interest.

One idea would perhaps to officially support/maintain some extension library, a bit in same way as WPF (Windows Presentation Foundation) has put up a WPF Toolkit where features trickles down after maturing to real .NET.

[Russell Keith-Magee]

Hyperbole in the ticket description aside, an AutoSlugField that sounds like a reasonable idea. The issue is finding an implementation that is acceptable. I'm not wild about the idea of an model save that will spawn a flood of queries back on the database trying to find a unique slug. This needs further discussion.

[Leon Matthews]

Replying to russellm:

I'm not wild about ... a flood of queries back on the database trying to find a unique slug.

For years I've used an approach that requires a single SQL query to find a unique slug value (with caveats, below). Treat the 'first guess' of the slug as a prefix, and fetch a list of possible conflicts using a single LIKE query. The suffixes in that list can be processed to determine the next suffix to try.

Caveats:

1. LIKE queries are slow.

2. Race condition: Two or more clients racing to save their object with the same 'unique' slug. The database can handle this if the column has a unique constraint: The loser(s) of the race need to catch an IntegrityError and try again.

3. Slug naming restriction. For unique slugs to work we have to conceptually break a slug into slug+suffix. This could be done by either separating them with a non-slug character, or enforcing a convention that, for example, numbers at the end of the slug are to be treated as a numerical suffix. I prefer the latter, but either policy might be uncomfortable for some users.

As a recent Python/Django convert, I lack the confidence to submit an actual patch. That said, if somebody is willing to supply a little hand-holding on how to write & run Django tests, I'd be willing to give it a go...

[Simon Litchfield]

Leonov's solution sounds good, as fallback in case the first guess doesn't work- in normal circumstances, the first guess will usually be OK.

This leaves us with 1 insert/update query most of the time; and 3 queries some times (1 failed insert/update, 1 like, and 1 successful insert/update). Only time you'd have more than 3 is in race condition.

[Mikhail Korobov]

Do we need this in django core? There is django-autoslug (​http://pypi.python.org/pypi/django-autoslug/) package. What's the benefit in including AutoSlugField in django itself over improving e.g. django-autoslug implementation?

django-autoslug is now spawning a flood of queries to find a slug but it supports custom slugify filters (django's slugify doesn't fit well for non-english web sites), FK-dependent slugs, non-universally-unique slugs (i.e. it may be ok to have similar slugs for articles with different years).

[Leon Matthews]

I think it's fundamental enough functionality that the core is the most appropriate place for it.

Pretty URLs are an important part of Django's philosophy. I think having at least basic autoslug functionality available 'out of the box' is valuable. Just think how much nicer (and search engine friendly) the URLs in the tutorial, and everybody's first blog app, will look... :-)

Perhaps the real question here is where the line between the core and 3rd party should be drawn? That's for the core developer's to decide, but my feeling is that Django should err on the side of 'batteries included'.

We already provide almost everything you need to create a great site, without any dependences, which is a great quality. Features that add to that quality, and fit most use-cases, should probably find their way into the core once they have settled down and become stable -- this seems to be the way Python itself is going with bringing code into its core library.

[Simon Litchfield]

Lets not overlook what's currently included, SlugField, with it's prepopulated_fields javascript, is an incomplete, client side only attempt at achieving what this field solves properly from both sides. Why not include the proper job, makes sense on all levels.

Typically we're already spawning 2 queries for a simple save. What's an extra zero or maybe 2 as fallback on duplicate slug? Developer's aren't forced into using it anyway. Those of us concerned about database scaling/performance usually have plenty of tuning to do whether this is available in core or not.

[Aymeric Augustin]

Change UI/UX from NULL to False.

[Aymeric Augustin]

Change Easy pickings from NULL to False.

[Aymeric Augustin]

The concept described in this ticket was accepted, the discussion is now about the implementation.

[F. Malina]

Just to revive this ticket. Here's an example implementation I find quite acceptable in user code:

class Article(models.Model):
    title = models.CharField(max_length=128)
    slug = models.SlugField(max_length=130)
    # ...

    def save(self, *args, **kwargs):
        if not len(self.slug.strip()):
            self.slug = slugify(self.title)

        _slug = self.slug
        _count = 1

        while True:
            try:
                Article.objects.all().exclude(pk=self.pk).get(slug=_slug)
            except MultipleObjectsReturned:
                pass
            except ObjectDoesNotExist:
                break
            _slug = "%s-%s" % (self.slug, _count)
            _count += 1

        self.slug = _slug

        super(Article, self).save(*args, **kwargs)

I've also used a regex based alternative implementation:

import re

class Article(models.Model):
...
    def get_slug(self):
         return '%s-%s' % (slugify(self.this), slugify(self.that))

    def check_slug(self, slug):
        # ensure uniqueness
        while(Article.objects.filter(slug__iexact=slug).count()):  # if not unique
            suff = re.search("\d+$", slug)  # get the current number suffix if present
            suff = suff and suff.group() or 0
            next = str(int(suff) +1)  # increment it & turn to string for re.sub
            slug = re.sub("(\d+)?$", next, slug)  # replace with higher suffix, retry
        return slug

    def save(self, *args, **kwargs):
        #...
        slug = self.get_slug()
        if not self.pk or (self.pk and not slug in self.slug):
            self.slug = self.check_slug(slug)
        super( ...

Could this be implemented by adding a pre_save method to the SlugField?
​https://github.com/django/django/blob/master/django/db/models/fields/__init__.py#L2073

Maybe call it add_auto_increment=True.

Any ideas on how this could actually look in the core, as opposed to just user code?