All very good questions. There is a lot of nuance and history here that short answers have to elide that is worth the research if one is interested.
Couldn't such leaks be avoided if the merchants didn't keep the details of online transactions? In other words, why payment processor like Visa or Mastercard can't act as a middleman?
Merchants are not supposed to store payment details following a transaction- except in rare cases where users are engaging with merchants to create a recurring transaction, e.g. a subscription- but some nevertheless do, in contravention of requirements they have to follow for taking card payments, colloquially called PCI after the organization that administers them (which is sponsored by the payment brands, Visa and Mastercard). These improper storage points have of course been compromised leading to significant fines and costs for the merchants involved.
However, storage, whether proper or improper, is far from the only attack point. There has to be machinery to capture the card details from the cardholder at the point of sale and communicate that with the various banks involved- at minimum the acquiring bank, the bank with whom the merchant has a relationship allowing them to collect cardholder payments- and cardholder banks, the banks responsible for approving the use of credit or confirming the availability of funds for debit. This communication is often logistically brokered by payment processor middlemen, which operate in various layers, specializations, and relationship shapes, depending on industry and so forth.
All players in the communication flow have to adhere to PCI requirements around "cardholder-data-in-transit" but as one might imagine they all do better or worse at this and all have been subject to compromise as well.
At any rate- the answer is that attackers go where the weak points are. There are many weak points in the overall payment machinery, so many that the Federal Reserve last year started a national program to systematically improve both security and latency in all payment systems (see https://fedpaymentsimprovement.org/).
That certain weak points have been exploited and there has been a recent history of public shaming is not an indication that there are not other weak points lurking in the system.
Why doesn't it work that way?
The mere presence of a middleman does not provide better security, though it can reduce the attack surface- or simplify the attacker's problem, depending on how well they do their job.
However, what you are really referring to is a better protocol- which can provide better security- and that's what we are seeing with the rise both of chip-based payments (called EMV in the US) and its online cousin tokenization. Both of which work by using cryptography to create single-use payment credentials at the point of sale, rather than always distributing a reusable payment credential. Lots to google there if interested.
Is a CSC (card security code, a 3 or 4 digit code printed on the card) always required when buying online and if not, what does it depend upon if it's required - is it up to merchant whether to ask for it?
It is not always required when buying online. The decision is usually the merchant's, which can experimentally weigh:
- reduction in sales due to consumers having to enter more data to complete a transaction
- reduction in fraud due to what might be loosely called an additional factor that is slightly less available to fraudsters
- reduction in fees from acquiring banks on transactions that use the CSC vs those that don't.
If they are always required, why is it such a big deal when millions of credit card details leak due to a hack?
It's a big deal from a news perspective because, for various reasons, more consumers pay attention to issues of this kind now. Largescale leaks from several years ago that were far more damaging in absolute terms received far less media attention.
It can also be a big deal for the retailers involved from a business perspective because the rules- called PCI, as mentioned earlier- have gotten tighter, and fines and penalties for violating the rules have gotten larger. So when retailers violate the rules, like storing cards when they should not have, the impact can now be materially significant to their business.
It's less of a big deal from an operational or fraud perspective, because all the parties are accustomed to the remediation- send consumers new cards, invalidate the old ones, rinse and repeat. Expiration dates have gotten shorter on newly issued cards, so loss of a large number of cards is to some extent just an extension of normal workflow for card providers.
Why the CSC number is only 3 or 4 digits? Couldn't fraudsters try all the possibilities? Or is there a maximum number of attempts before a card is blocked?
The code that performs system of record transaction processing checks for various indicators of real or attempted fraud, including multiple attempts against the same payment instrument within a particular time frame.
There is also a lot of geographical rationalization- use of the same card number in two physical Point of Sale systems in different geographical areas in the same day will get heightened scrutiny.
But there are many other signals here- payment fraud is a very interesting and active application area for machine learning.
Why is the CSC number even printed on card? Wouldn't it be better if there was just one password not written on the card that is used to authenticate both types of transactions?
Convenience, all around.
It is absolutely fair to say that the CSC number does not rise to the level of being a second factor, but it is also absolutely fair to say that a second factor is still infeasible in the real world when it comes to usability and convenience. Even security people still struggle with second factors, much less ordinary consumers.
A more common "second factor", so to speak, is required use of cardholder zip code, which is much more common than required use of a CSC.
My sense is that on balance, CSC is attributed with a relatively modest reduction in fraud at relatively modest infrastructure expense, but that it has not been considered a success. Chip payments and tokenization are much bigger impact.
When I read headings like "10 millions of credit card numbers leaked" do they actually mean only the numbers? Isn't the numbers alone useless without data like expiration date, name of the card holder, which as far as I know are also used to authenticate a transaction?
A reported compromise will be of whatever the compromised party considered to be sufficient data to capture payments. Card number and expiration date are the only data points that are always required by payment processors and other downstream systems. Other data points like name, address, zip code, phone number, CSC, email may or may not be, depending on the particular details of the merchant, the business, their provider relationships, and so forth.
Most of the use cases around additional data collected at payment time are for marketing rather than strictly for payment verification.
