Support EMAIL_SSL_CERTFILE for private certificate authority

[dkaylor]

We have an SMTP server that is not signed by a public CA. Sending email with no SSL settings results in an "SSL: CERTIFICATE_VERIFY_FAILED" error.

If we set EMAIL_SSL_CERTFILE, we receive the same error. We do not have access to the key file to test with EMAIL_SSL_KEYFILE. Clients often do not have access to keys so this shouldn't be required.

Django is loading the cert files with load_cert_chain, but I believe load_verify_locations would be more appropriate:

​https://github.com/django/django/blob/main/django/core/mail/backends/smtp.py#L63

The examples in the Python docs use the former for servers and the latter for clients:

​https://docs.python.org/3/library/ssl.html

I wrote a simple test with load_cert_chain and it fails with the same SSL error:

ssl_context.load_cert_chain(cacert)

If I change to load_verify_locations it works

ssl_context.load_verify_locations(cacert)

[Mike Edmunds]

This makes sense to me, but Python's SSL/TLS is a little outside my expertise. It would be good to get Mariusz's input.

Django 4.2 changed to use ssl.create_default_context() if neither certfile nor keyfile is set. This enables certificate validation and hostname checking, and is a Python ​ssl security best practice.

I wonder if we shouldn't also be using ssl.create_default_context(capath=...) when an EMAIL_SSL_CERTFILE is provided, for exactly the same reasons? Followed by load_cert_chain() when necessary. (This would require a release note ​similar to the one in 4.2.)

See also ticket:34550 and ​this StackOverflow answer.

[Sarah Boyce]

​In the 4.2 release notes:

EmailBackend now verifies a hostname and certificates. If you need the previous behavior that is less restrictive and not recommended, subclass EmailBackend and override the ssl_context property.

Sounds like a duplicate of #34504

[Mike Edmunds]

I don't think this is a duplicate of #34504. That ticket wanted to disable hostname checking and certificate verification, which is indeed not recommended.

This ticket is trying to use a private certificate authority with hostname checking and certificate verification enabled. I'd think we'd want to encourage that when using a private CA. Django's SMTP EmailBackend makes that difficult right now, by requiring subclassing and overriding an undocumented property.

I would treat this as a feature request for Django's SMTP EmailBackend to support setting EMAIL_SSL_CERTIFICATE to a private CA or self-signed certificate, with all recommended security enabled.

If we don't want to do that, we should probably add some documentation along the lines of:

• EMAIL_SSL_CERTIFICATE is meant only for client authentication, and therefore must either include the private key or be used together with EMAIL_SSL_KEYFILE.
• To use a private certificate authority or self-signed certificate with your SMTP server, don't use Django's EMAIL_SSL_CERTIFICATE. Instead, add your private CA to your system's OpenSSL ca-path or set the SSL_CERT_FILE and/or SSL_CERT_DIR environment variables to point to it. (See Python's ​ssl.get_default_verify_paths().)

(Also, I might be misunderstanding, but it looks like when EMAIL_SSL_CERTIFICATE is set the ​SMTP backend creates a less-secure ​SSLContext with checking disabled. And I see we've actually ​documented that behavior.)

[Mike Edmunds]

(Also, since the docs for EMAIL_SSL_CERTIFICATE already say "certificate chain file," it seems reasonable to expect that could be a private CA bundle. Which would make this a bug.)

[Igor Scheller]

I added a ​PR to implement it as an additional feature (altho changing the current behaviour might be an option too)

[Igor Scheller]

As noted in the PR discussion, adding another option should be avoided in favour of the upcoming EMAIL_PROVIDERS setting, discussed in #35514.
The PR now reflects that change by allowing the setting to be set in the constructor and later from above providers config.

[Mike Edmunds]

This seems like a useful addition, given that:

• Internal private CAs are not all that exotic.
• Django's current documentation seems to suggest that EMAIL_SSL_CERTIFICATE can be set to a private CA bundle, but this doesn't actually work.
• Although the problem can be solved by subclassing smtp.EmailBackend to override ssl_context, that seems to be error prone. A lot of high-ranking solutions disable certificate checking entirely or introduce other security issues. (Another common recommendation is downgrading to Django 4.1.)

Question: am I understanding correctly that the proposed ssl_cafile option would also work to securely verify self-signed certs? (That seems like another semi-common Django email question that generates a lot of less-secure answers.)

[Igor Scheller]

Yes, it allows you to add a ​root-CA certificate / bundle of certificates which is used as the trust anchor, it can either be your own self-signed one or the one your organisation set up as a private CA.