When does S/MIME require the other party's certificate and when is their key enough?

Question

Trying to understand S/MIME implementation, I see that sometimes you must have the other party's certificate, and sometimes its enough to just have their public key. I'm confused as to when you need a certificate and when they key is enough. Can anyone clarify?

(I'm trying to understand the internals here, so my question is conceptual - I know that practically speaking, most email clients use the full certificate).

score 1 · Accepted Answer · edited Oct 07 '21 at 06:47

S/MIME builds on CMS (the standard name for what was previously known as "PKCS#7"). An encrypted email really is a CMS enveloped-data object. The bulk of the data is symmetrically encrypted, with a key which is derived/exchanged using one of the transport mechanisms detailed in section 6.2:

KeyTransRecipientInfo is basically asymmetric encryption with the recipient's public key. That's what is used when the recipient has a RSA key fit for encryption (i.e. not a RSA key tagged in his certificate as "signature only").
KeyAgreeRecipientInfo is for key exchange algorithms which are not asymmetric encryption. You will use that if the recipient has a Diffie-Hellman key.
KEKRecipientInfo is used when there is a shared (high entropy) symmetric key between sender and recipient.
PasswordRecipientInfo is used when there is a shared password between sender and recipient (which thus calls for some extensive password hashing to make up for the inherent weakness of the password).
OtherRecipientInfo is for all other cases, which did not fit in the categories above.

The critical point is that for both KeyTransRecipientInfo and KeyAgreeRecipientInfo, the recipient's public key (be it RSA, Diffie-Hellman...) can be identified in two ways:

  RecipientIdentifier ::= CHOICE {
    issuerAndSerialNumber IssuerAndSerialNumber,
    subjectKeyIdentifier [0] SubjectKeyIdentifier }

  KeyAgreeRecipientIdentifier ::= CHOICE {
    issuerAndSerialNumber IssuerAndSerialNumber,
    rKeyId [0] IMPLICIT RecipientKeyIdentifier }

  RecipientKeyIdentifier ::= SEQUENCE {
    subjectKeyIdentifier SubjectKeyIdentifier,
    date GeneralizedTime OPTIONAL,
    other OtherKeyAttribute OPTIONAL }

  SubjectKeyIdentifier ::= OCTET STRING

When the public key comes from a certificate, then the public key can be identified using either "issuerAndSerialNumber" (a copy of the "issuerDN" and "serialNumber" fields of the certificate), or with the "subject key identifier". The latter is a bunch of bytes which can optionally be included in a certificate (as a Subject Key Identifier extension). Either information is readily extracted from the certificate. You never need the complete certificate, but you need parts of it.

S/MIME also supports public keys which are not obtained from certificates, but from "S/MIME preferences". The idea is (or maybe was ? I haven't seen that in a long time) that a user may have only a signature certificate (e.g. with a DSA key). In that case, that user could generate his own key pair for asymmetric encryption or key exchange, sign the public part, and include it in messages he sends. People who want to respond can then use that public key for key transport or key agreement. Since that key is not contained in a certificate, it cannot be identified with fields from the certificate. Instead, it will be identified with a "key identifier" similar to the "subject key identifier", i.e. a bunch of bytes.

Summary: all the crypto uses only the key, but the message format must include a designation for the key which was used, and that designation will use fields extracted from the recipient's certificate (if the recipient's public key indeed comes from a certificate).

When does S/MIME require the other party's certificate and when is their key enough?

1 Answers1