How does the SSL connection protocol work with self-signed certificate?

Question

I'm using the self-signed certificate, but I don't know how this protocol works. I connect two apps with a socket SSL and it works fine. The server is a Python app and the client is an Android app. I created the self-signed certificate with openSSL, now I have two files: the private key and self-signed certificate. The server uses all files but client only uses self-signed certificate.

I found many pages explaining the protocol connection with signed certificate and CA, but none with self-signed without CA.

Protocol with signed: Client Server

ClientHello                  -------->
                                                ServerHello
                                               Certificate*
                                         ServerKeyExchange*
                                        CertificateRequest*
                             <--------      ServerHelloDone
Certificate*
ClientKeyExchange
CertificateVerify*
[ChangeCipherSpec]
Finished                     -------->
                                         [ChangeCipherSpec]
                             <--------             Finished
Application Data             <------->     Application Data

(This schema has been shamelessly copied from the RFC.)

How about with a self signed certificate?

Answer 1

In the SSL protocol (now called TLS), the certificates are a black box: from the point of view of SSL, the certificates are received from the server, and somehow the public key to use becomes known to the client. The certificates are a vessel used to convey the server's public key to the client, and all the CA business is a way for the client to gain some assurance that the received public key is indeed the correct one from the intended server.

When the server uses a self-signed certificate, the client cannot transfer any trust from a known CA in order to ascertain that the received self-signed certificate is the genuine thing, and not an imitation from some active attacker. However, if the client already has knowledge of the server's public key (e.g. during some prior installation phase, under controlled conditions, the server's self-signed certificate was shown to the client, and the client remembers it), then the client, by definition, knows the right server's public key, and uses it.

In any case, the SSL-level protocol is completely unchanged. The server stills sends a Certificate message, and the client still uses the server's public key for its ClientKeyExchange message.

Answer 2

EDIT: See here: How does SSL/TLS work? Or Here Self-signed cert - how it works

I don't undertand, when I signed the certificated by CA the protocol is this: client send Hello to Server--- Server send hello and certificate to Client--- Client verify certificate with a CA and send confirmation to Server--- Finally Server send confirmation signed to Client. – juve164 Jun 14 '14 at 12:15

What happens with a self-signed certificate. Client send Hello to Server --- Server send certificate to Client and Client verify the certificate, but how??? compare if it is the same certificate??? – juve164 Jun 14 '14 at 12:23

Let's put some perspective on this. Certificate user Public-key Cyrptography. This means that a public key and private key are generated simultaneously and are mathematically joined. Only this private key works with this public key and vice-versa.

When you introduce a certificate, it is nothing more than a "piece of paper with a signature" on it. That signature is from the private key. This just provides a reference for what the key is used for; aka google.com.

TLS Handshake:

C ClientHello: This is the beginning, we're talking to a server so we can start the TLS process.

S ServerHello: The server responding back to acknowledge we're talking in TLS now.

S Certificate: This is a copy of the certificate that is installed on the server.

S ServerKeyExchange: This is part of the protocol used to allow both devices to arrive at the same symmetric key to use after the TLS handshake

S CertificateRequest: Only used if the server wants to verify the client with a certificate. This can be conveyed as a form of authentication.

S ServerHelloDone: The server is done with it's part of the TLS handshake.

C Certificate: Only provided if needed; see CertificateRequest above.

C ClientKeyExchange: This is the companion to ServerKeyExchange to make sure both sides have a symmetric key.

C CertificateVerify: Part of CertificateRequest and Client-Side TLS.

C ChangeCipherSpec: This is usually the indication that we're done with everything and were' ready to start talking with encryption.

C Finished: This is the end of the client side.

S ChangeCipherSpec: The server is agreeing to the cipherspec.

S Finished: It's all done and we're fully encrypted now.

In this process, the certificate being sent by the Server is going to be what's configured on the server. If this is signed by a CA, typically this would include the leaf certificate (the one with the domain name), it's immediate signing authority and the parent of that signing authority (and so on if applicable). So Certificate would typically include 3 physical certificates.

The client then looks at the issuer on the leaf certificate and starts following the chain back up. The Intermediate has an issuer (being the Root CA) and so it would check that certificate. The Root CA is a Self-Sign certificate. It's a certificate where it's own Private Key signed the certificate -- not some other entity. As long as the client trusts the Root CA, it will trust the intermediate, which in turn means it will trust the leaf.

With a self signed certificate on the Server, there is only certificate issued for the client to verify. It will look at the issuer of the certificate and see that it was signed by itself. As long as the client trusts that CA, it will trust the connection and continue.

All Root CA's are self-signed. The only reason your computer trusts them is because their configured on your computer to be trusted.

https://en.wikipedia.org/wiki/Transport_Layer_Security#TLS_handshake

Answer 3

From Wikipedia: https://en.wikipedia.org/wiki/Self-signed_certificate

In cryptography and computer security, a self-signed certificate is an identity certificate that is signed by the same entity whose identity it certifies. This term has nothing to do with the identity of the person or organization that actually performed the signing procedure. In technical terms a self-signed certificate is one signed with its own private key.

Your private key is your CA.