Warn about invalid RSS characters in syndication docs

[Michael Wood]

I have some data which comes from log files that I'd like to put into a RSS feed, unfortunately due to the nature of this data it sometimes contains control characters e.g. \0001 \0003 , this causes it to fail RSS feed reader validation due to these characters (although valid utf-8) are not allowed (1).

I'm not sure if this is something that should be fixed in this module, perhaps in sax/saxutils or in somewhere like django.utils.encoding force_text ?

At the moment I'm working around this issue with a regex which replaces this range of chars.

(1) ​http://www.w3.org/TR/REC-xml/#charsets

[Tim Graham]

We could look and see if other web frameworks perform sanitization or make alternate recommendations. If we don't make a change in Django, we could at least update the docs to note that requirement of sanitizing your own input and make a recommendation of how to do so.

[Claude Paroz]

#20197 is similar but targets XML serialization with dumpdata. I just added a patch in that ticket to loudly fail instead of silently producing invalid XML. Automatic sanitation is tricky, because depending on the use case, you might want to remove the offending chars, replace them with some alternative coding, or simply fix the source.

The patch for #20197 also affects RSS production, as the same django.utils.xmlutils.SimplerXMLGenerator is used. If it gets committed, we might want to add a similar admonition in syndication docs.

[Claude Paroz]

Proposal for a documentation addition:

docs/ref/contrib/syndication.txt

@@ -919 +919 @@
     ``self.feed`` for use with `custom feed generators`_.
 
     All parameters should be Unicode objects, except ``categories``, which
-    should be a sequence of Unicode objects.
+    should be a sequence of Unicode objects. Beware that some control characters
+    are `not allowed <http://www.w3.org/International/questions/qa-controls>`_
+    in XML documents. If your content has some of them, you might encounter a
+    :exp:`ValueError` when producing the feed.
 
 :meth:`.SyndicationFeed.add_item`
     Add an item to the feed with the given parameters.

[Tim Graham]

exp -> exc, otherwise looks good.

[Claude Paroz <claude@…>]

In 1c90a3dc:

Fixed #24985 -- Added note about possible invalid feed content

Thanks Michael Wood for the report and Tim Graham for the review.