In asymmetric cryptography, everyone has the public keys and only the entity being verified by the public key has the private key. The public key and private key are related in a way that having the public key doesn't reveal anything about the private key, but that the holder of the private key can prove their identity by having someone verify a challenge using the public key.
Basically, something that is encrypted with a public key can only be decrypted by the private key and something that is encrypted with the private key can only be decrypted with the public key.
To prove the server is who they claim to be, they encrypt a challenge with their private key (which only they have) and the public key is used to verify that the challenge can be decrypted, thus the other party has the private key.
To talk to the server securely, the public key is used to encrypt a symmetric session key. That session key is encrypted with the public key and sent to the server. Only the valid server (that has the private key) can decrypt the session key, so only the client (who created the session key) and the server with the private key can communicate.
The public key is trusted by the client because it is similarly signed by a root certificate authority which is included with the browser.
Update: I misread the question as SSL, not SSH. The basic idea is still roughly the same, but the server certificate often won't be signed by a CA, in which case the certificate needs to be manually verified the first time after which your client should keep track of who you connected to last time. The key exchange also differs in terms of how a key is agreed upon, but I don't know the SSH handshakes as well as I know SSL's, so see Thomas Pornin's answer for that.