Authentication process between the server and the browser

Question

How the browser ensures that it is communicating to the right server?

Answer 1

TL;DR: Authentication of the server happens only with HTTPS, but neither HTTP or other protocols supported by the browser. Without proper authentication harmful attacks like sniffing or traffic manipulation are easy.

---

In general the browser makes use of functions of the underlying operating system and infrastructure to lookup the IP address for the domain given in the URL (DNS lookup). It then connects to this IP address and exchanges the data.

This process can go wrong on several places. For example an attacker could tamper with the DNS lookup so that the IP returned by the lookup is not the IP address of the intended server but of an attacker controlled system. An attacker could also tamper with the routing of the traffic so that any traffic intended to the target's IP ends add a system the attacker controls or passes thru such system.

By doing this an attacker could achieve passive sniffing of the traffic for example to steal credentials like passwords, active traffic manipulation as man in the middle for example to inject ads or malware delivery or the attacker might fully incorporate the target system to serve different content.

Normal HTTP traffic (and also FTP traffic) does not provide much protection to this. There might be DNSSec used in the network to prevent DNS spoofing but DNSSec is not employed for all domains and it is rarely enforced, i.e. most setups will accept unsigned (and thus maybe manipulated) DNS responses. And even if DNSSec is used to guarantee the proper target IP address attacks against the routing could still be done.

Only when using HTTPS (i.e. https://.. URL's) a proper authentication of the server is done followed by an encryption of the traffic between browser and server. If done properly this prevents all of the described scenarios, i.e. sniffing, active man in the middle and incorporating the identity of the target no matter how exactly the attack is done.

One essential point here is the proper authentication of the server during the initial TLS handshake (before exchanging application data) which is based on checking certificates. There is no need to explain again how this is done since there are already excellent answers about this on this site, like SSL Certificate framework 101: How does the browser actually verify the validity of a given server certificate?.