EDIT - OP really did mean use of Server Certificate. Answer changed to reflect this. Mostly, stuff added to end.
Both answers (@mti2935 & @MrNerdHair) address your questions, however, one of your comments says you're still a bit confused about the use of the Private Key in the TLS exchange. I'm providing this answer to address that by outlining how the TLS protocols work at a high level. After the protocol outline, we can examine the contribution of the Server's Private Key and why having just the Public Certificate (and hacking DNS) is insufficient to faking a web server.
Methods for Key Establishment
Let's examine four protocols: RSA, Diffie-Hellman and how TLS uses each of them. First of all, every network cryptographic establishment protocol ends with the creation of a secret session key used for symmetric encryption. This is an engineering principle and not a security principle. Why? Because symmetric cryptography is designed to be fast while other forms of cryptography (RSA) are slower and some key agreement protocols don't have cryptography. Protocols 2, 3 & 4 show how both sides securely agree upon a session key.
RSA
RSA Public Key Cryptography can be one-way cryptography or two-way cryptography. However, it differs from symmetric cryptography in that neither party in the exchange needs to have met the other party prior to the exchange and exchanged a secret. Focusing on one-way cryptography, how can something (like a message or key) be sent encrypted?
- Cert_Server contains the public key of the server: (N,e) where N=pq.
- Private Key of the server is (N,d), although it can keep p, q and a Phi(n) and it needs its Certificate to give out to clients. No one should have the private key but the server.
Here's how a client sends a secret message to a server:
Client to Server: Hello
Server to Client: Hello, Cert_Server
Client, internal: Somehow Validate Cert_Server and, if valid,
Client to Server: Enc(plaintext, Key_Public)
Server, internal: Dec(cryptext, Key_Private)
That's it, the Client encrypted plaintext and sent it to server. Everyone can see the cryptext (Enc(msg, key_pub)), no one but the server can see the plaintext without breaking RSA by factoring N or stealing the server's private key (d which was computed knowing primes p and q). Server can see the plaintext by decrypting using its Private Key (d,N). The Client's Validation step involves checking that the cert is internally valid (all signatures check out), checking the Root CA, checking revocation lists.
Diffie-Hellman
Briefly, in DH Key Exchange, we start with two numbers (P,g), P is Prime (actually a Safe Prime, where P=2Q+1 and Q is also Prime) and g is a generator.
The exchange may look something like this:
Client to Server: Hello
Server to Client: Hello, (g,P)
Client, internal: A = RND(P), a = (g^A)%P
Client to Server: a
Server, internal: B = RND(P), b = (g^B)%P, K = (a^B)%P
Server to Client: b
Client, internal: K=(b^A)%P
At the end of the exchange, both parties know K, however, no one watching the exchange which contained (g,P,a,b) can compute K without being able to perform a discrete logarithm in the field P.
Note, this is subject to man-in-the-middle attack. Because (g,P) are known (sent by the server). That means, Mallory can step into the middle and run the server side of the exchange with the client and run the client side of the exchange with the server, produce two separate secrets (K) and neither side will know Mallory sits in the middle.
TLS & RSA
The TLS cipher suite that uses only RSA works as follows (lots of other bits snipped):
Client to Server: Hello
Server to Client: Hello, Cert_Server
Client, internal: Somehow Validate Cert_Server and, if valid,
Client, internal: K=RND()
Client to Server: Enc(K, Key_Public)
Server, internal: Dec(cryptext, Key_Private)
Notice that this is essentially the same as our description of RSA above, except that for the plaintext we've used a client generated session key, K. Note, some of the problems with TLS_RSA cipher suite is that the client generates the K. Often, the client doesn't do a good enough job generating K (cryptographically strong RNG), leaving the session open to attacks.
TLS & DH & RSA
TLS cipher suite for DH_RSA (with many bits snipped out, order could be slightly broken):
Client to Server: Hello
Server, internal: B=RND(P), b=(g^B)%P
Server to Client: Hello, Cert_Server, g, P, Sign({b}, Key_Private)
Client, internal: Somehow Validate Cert_Server and, if valid,
Client, internal: A = RND(P), a = (g^A)%P, K = (b^A)%P
Client to Server: a
Server, internal: K=(a^B)%P
Here we have the merging of both protocols. We fix the MITM attack DH Key Exchange suffers by signing the DH ephemeral, b, from the Server. The client and server then use DH Key Exchange to produce the session key, K.
Answer to Theft Question
OK, so after that long interlude, what about use of a Server's Certificate and a DNS hack to faking (with security) a legitimate web site?
Go back and have a look for Private_Key in three of the four protocol descriptions above. You'll see that in each case, the Server is either decrypting some message with it or signing some message with it. So, unless the attacker has stolen the Server's Private Key, he cannot impersonate that Server by running the server side of each of these protocols. The client may detect the failure of the Server by a number of means. I detail each detection for the algorithms below.
- RSA - Strictly speaking, the client cannot detect the server failing to have the private key in this case. If all the client ever does is RSA encryption with the Server Public Key, the client will never know the Server isn't legitimate. OTOH, the fake server cannot decrypt the data, so this isn't the worst situation, although it's not optimal. However, and this is a big however, no communication system would be built s.t. client sends server messages without a server ever being challenged. That is, the server must prove its knowledge of the private key at some point before the client trusts it. So, the fake server will be detected eventually, if not exactly within this algorithm.
- DH - DH Key Exchange - Ignoring the MITM attack, here, the session key, K, a.k.a. Master Session Key, is used to generate a number of other keys for Privacy (encryption) and Integrity (hmac). In the next step the client and server use these generated keys to challenge each other to prove they know the keys. This amounts to correctly encrypting and hmac'ing data and sending it back and forth. I will show what that looks like below.
- TLS & RSA - Same as DH, next step, mutual proof of Keys.
- TLS & RSA & DH - There are two touch points here and an attack I want to highlight. First, if somehow we make it to the DH Key exchange step, both sides would do mutual proof of Keys. However, notice the
Sign(b, Key_Private) step? A fake server without the Private Key couldn't perform this signature, so the client would know (because it should check) at that point that the server is fake. However, there's an attack here. Attacker can run this cipher suite with the real server, which will send it a signed ephemeral! That is, the fake server could fetch (one time only needed) a copy of Sign(b_old, Key_private). The client validates the signature and everything looks good. This is why the client must eventually authenticate the server using the generated session key.
Finally, sample mutual proof of keys a.k.a. client and server mutual authentication:
... continuing exchange ...
Client, internal: K_enc = F1(K), K_mac = F2(K), Nc = NONCE
Client, internal: Ec = Enc(Nc, K_enc), Hc = HMAC(Nc, K_mac)
Client to Server: Ec, Hc
Server, internal: K_enc = F1(K), K_mac = F2(K), Ns = NONCE
Server, internal: Hc == HMAC(Dec(Ec, K_enc), K_mac)?
Server, internal: Es = Enc(Ns||Nc, K_enc), Hs = HMAC(Ns||Nc, K_mac)
Server to Client: Es, Hs
Client, internal: Hs == HMAC(Dec(Es, K_enc), K_mac)?
Please note the two internal steps, one for the server and the other for the client which look like this: Hs == HMAC(Dec(Es, K_enc), K_mac)? This is the mathematical proof that client runs to prove the server knows the session key, could produce the encryption and hmac keys, and was able to decrypt the Nonce, Nc, it sent over (server concatenated to Ns).
That specific check breaks down as follows:
- Decrypt Es using K_enc. Plain text is Ns||Nc.
- Check that plaintext Nc matches the Nc client first created.
- HMAC(Ns||Nc, K_mac). Run our own version of HMAC on Ns and Nc.
- Check that HMAC == Hs. If equal, all good.
All of these steps constitute a proof of the various steps that came before and allow the server to prove to the client it is legitimate.
I hope that answers your question.
