Why is the HTTP Server header not used by most servers, but the User agent used by most browsers?

Question

I've read that the Server header would disclose information, and that its use would be bad. The arguments made in that question also apply for the User agent header. So why is it still used, and not a "dummy", like the one google uses for the server header field?

As I understand it, the user agent was useful in the days browsers were still heavily buggy, and would render very different results. To a certain extent, this is also true today, but certainly has decreased. Is there no other method to know whether a browser sais "I'm mobile" than the user agent? Also, user agents are a mess. The browser vendors tried to trick stupid user agent recognition scripts, because of which the Chrome user agent reads like a list of relevant browsers and rendering engines.

So why aren't they abandoned?

Answer 1

User-agent strings do provide the companies running web servers with useful information. If they're contemplating introducing web site features or content they want to know if the majority of their user base have browsers that support that technology. They can look at the user-agent strings logged in the past month or two and get a sense of what browsers are in use. It can also tell them useful info like whether the user is connecting via a mobile device versus a PC, which can be used to customize the user experience.

Conversely, I'm not aware of any purpose served by web servers revealing their product or version other than to aid third-parties (e.g. Netcraft) who collect and summarize such information. The user never sees the server type and the browser doesn't act any differently depending on the server banner (that I've ever seen).

Since Internet web servers are always running and publicly accessible that allows attackers to scan them quite easily looking for signs of vulnerability. A site shouldn't rely only on the obfuscation of what web server and version they are using for security, but it can help to make them a less likely target. Since it doesn't really disturb anything to disable the server banner it is an easier change to introduce.

While there is risk to the user of disclosing that they're using a specific browser that may be vulnerable to specific attacks, an attacker can't scan the Internet looking for vulnerable browsers. A user would have to send a request specifically to an attacker controlled server for them to collect this information. So there can be less risk associated with disclosing this information about a browser.

I'm sure the user-agent could be abandoned in favor of a 'feature string' or some other mechanism for communicating information about what the browser is and can handle to the web servers. But these too could probably be fingerprinted by attackers who could guess the likely browser based on the supplied feature list.