How exactly is RSA used in SSH

Question

I can find oodles of websites that tell me the server retains the public key and the client retains the private key. But nothing on how the client authenticates to the server and - perhaps - how the server authenticates to the client

Answer 1

Public key encryption is how it works. (The key algorithm makes little difference here)

Basically the following happens:

1. The user connects to the client with a “hey, can I connect please?” Request.
2. Server sends back “Ok, here is my pub key (hash) and these connections I support.
3. Exchange a key exchange with any of the key exchange algorithms.
4. Exchange a package which is encrypted with the private key that is decoded with the public key of the user that is know by the server. (Package holds a shared secret for the next level of encryption.)
5. If the user is authenticated through any of the possible means, continue to a shell / other program. Otherwise exit.

If you need a better picture check the RFC 4252.

(Also you can start a exchange in verbose mode with the -v flag… use 3 of them for maximum verbosity. )