Add search rank cd function and normalization for Postgres full text search

[Andrii Soldatenko]

It would be great to have cover density ranking for the given document vector and query function ts_rank_cd. It helps to order search results with different normalization strategies.

Also add normalization parameter option that specifies whether and how a document's length should impact its rank.

0 (the default) ignores the document length
1 divides the rank by 1 + the logarithm of the document length
2 divides the rank by the document length
4 divides the rank by the mean harmonic distance between extents (this is implemented only by ts_rank_cd)
8 divides the rank by the number of unique words in document
16 divides the rank by 1 + the logarithm of the number of unique words in document
32 divides the rank by itself + 1

Now you can do something like this:

class SearchRankCD(SearchRank):
    function = 'ts_rank_cd'

    def __init__(self, vector, query, normalization=0, **extra):
        super(SearchRank, self).__init__(
            vector, query, normalization, **extra)

query = SearchQuery('messenger')

Application.objects.annotate(
    rank=SearchRankCD(
        F('search_vector_title'), query,
        normalization=2)  # 2 divides the rank by the document length
).filter(search_vector_title=query).order_by('-rank')

[Simon Charette]

​https://github.com/django/django/pull/12597

[Mariusz Felisiak <felisiak.mariusz@…>]

In 0b51a4f:

Fixed #28194 -- Added support for normalization and cover density to SearchRank.