GitLab

On macOS, there are two different trust settings granting positive trust -
kSecTrustSettingsResultTrustRoot (root is explicitly trusted) and
kSecTrustSettingsResultTrustAsRoot (non-root is explicitly trusted as if it
were a root). The former "should" only apply to self-signed certificates,
while the latter applies to those not self-signed (which includes
self-issued certificates).

The existing code was checking whether or not a certificate was self-signed,
in order to determine which SecTrustSettingsResult it should be expecting.
However, that also meant checking potentially weak signature algorithms on
roots, which was failing, and thus causing the certificate to be treated as
untrusted. As the signature algorithm on a root certificate should not
matter, this change now only checks for self-issued (subject == issuer),
rather than self-signed.

This isn't 100% in line with Apple's documentation for these values, and is
thus a bit of a hack. However, this should be OK, as the Apple APIs to set
trust settings validate the consistency between configured trust setting and
the type of certificate it is, preventing invalid trust records.

This also removes the now-unused x509_util_mac IsSelfSigned function, as
this was the only usage.

Bug: 410574
Change-Id: Id7e12ed4f92da9cb4e0b8decf048e3cfc77bb06e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1574879Reviewed-by: Ryan Sleevi <rsleevi@chromium.org>
Commit-Queue: Matt Mueller <mattm@chromium.org>
Cr-Commit-Position: refs/heads/master@{#652570}