14

Two organisations I frequently deal with (including my bank) ask me my password to authenticate myself with them... well... they ask for "random" characters from my password.

As I understand it, this means that the password cannot be being stored salted and hashed? It has to be being stored in a retrievable format. Whether that's plaintext or some way of reversible encryption.

My question is whether this can be justified? Is this really a better idea?

In my mind the benefits are:

  • replay attacks are "harder" (hah!)
  • customer hopefully realises phishing requesting entire password is phishing

But is that really good enough to offset the big disadvantage that it is technically very, very possible for your entire password store to be obtained?

Pricey
  • 240
  • 1
  • 9
  • 3
    I've seen http://security.stackexchange.com/questions/4830/how-do-some-sites-e-g-online-banks-only-ask-for-specific-characters-from-a-pa but hope this is a different enough question. – Pricey May 30 '13 at 19:32
  • 3
    There is no overarching reason that outweighs the *considerable* problems with unsalted unhashed passwords. As for hoping the customer realizes anything, that's a false hope. People are human. – Jonathan Garber May 30 '13 at 19:42
  • 1
    90% of hashed and salted passwords are very easy to obtain. http://arstechnica.com/security/2013/05/how-crackers-make-minced-meat-out-of-your-passwords/ "Hashed and salted" is not a magic bullet. – Xander May 30 '13 at 22:23
  • @Xander I don't think the hashes cracked in the linked article were salted? Also md5... which isn't exactly recommended? – Pricey May 30 '13 at 22:37
  • @PriceChild You're right they weren't, but for a weak hash algorithm, the addition of a salt is essentially useless. Yes, MD5 is not recommended, but most of the systems out in the real world that store hashes of salted passwords are using algorithms that are not recommended, MD5 or SHA-1 specifically. So, in the real world, most salt+hash implementations are more security theater than security, so claiming that's the "right" way to store passwords is just too simplistic. – Xander May 30 '13 at 23:18
  • 2
    @Xander: Which was the point I was making with my comment "weak salt/hash is worse that good encryption." – Alan May 31 '13 at 00:29
  • 1
    Two of my banks require as part of my login: account name, password and 3 chars from a secondary password - which I hope is stored in a separate database. This level of security works for me:-) – Rory Alsop May 31 '13 at 10:16

5 Answers5

18

Security is a complex endeavor, and knee-jerk reactions like “plaintext password? never” are just as bad as “passwords must contain special characters” or “my crypto's bigger than yours” or “if I can't understand it nobody can” if you don't consider the full context.

First, let's analyze the assets. There are two subjects: the bank, and the customer. They hold several assets:

  • The bank's money. Depending on what the attacker do, the bank may need to spend money fixing problems, to compensate the customer or other parties, and may see its reputation tarnished.
  • The customer's money (and his private information).
  • The customer's password. If compromised, he will need to change it, perhaps on other sites as well.
  • The authentication data sent during one session. This can be different from the password, for example in the case considered here this is (typically) two random characters from the password.

The bank is in full control of the security architecture. It would be unsurprising for it to privilege the security of its own assets over the customer's. This doesn't bode well for the password.

There are many threats around the password, including (numbering for convenience, not to suggest any particular order of importance):

  1. An attacker gains access to the bank's live system or to backups and is able to access the password the way it is encoded in storage. The attacker would almost certainly be able to obtain a large number of passwords at once this way.
  2. An attacker gains access to the computer used by the customer (e.g. by installing malware on it) and grabs the authentication data, possibly hijacking the session.
  3. An attacker obtains the authentication data by shoulder surfing or other method that doesn't give him control over the session.

Hashing the passwords on the bank servers exclusively mitigates threat #1, unauthorized access to the encoded password. This mitigation is limited as it is only effective against sufficiently strong passwords; weak passwords will fall even if protected by a slow hash.

Splitting the authentication data from the password increases the vulnerability to this threat. On the other hand, it mitigates threat #3 and to a small extent threat #2: an attacker in possession of the authentication data for one session cannot easily reproduce it to open another session. If he makes many attempts until he gets a challenge that matches the authentication data he has, his attempts are likely to be blocked by the bank's server. Eventually, if he watches enough successful authentications, he'll get the whole password, but he risks getting noticed.

At this point, the bank needs to weigh the risk and damage from threat #1 with the risk and damage from threat #3, and to consider alternative ways of mitigating threat #3. There are numerous authentication methods and I won't go into details here; I'll concentrate on threat #1.

If threat #1 leads to an attack, the attacker is in possession of a large number of hashed or non-hashed passwords. Keep in mind that hashing only makes a difference from sufficiently strong passwords. The attacker is unlikely to be able to attack all the accounts anyway, because after a while he will be detected (maybe not with 100% accuracy, but enough to slow him down) by the bank's fraud pattern detection. So he's content with focusing on the accounts with the passwords that are easiest to crack; hashing doesn't slow him that much. The mitigation offered by hashing isn't very good in this scenario.

If you're a user with a strong password, you would understandably prefer not to be affected by the attack — let the troubadours bear the brunt. The bank has a different outlook: it doesn't care whose account gets breached, only about the number of breached accounts. If at this point you get huffy about banks not caring about geeks who take their security seriously — do you have a credit card? Merely having one allows anyone to make a purchase charged to your account by rattling off a 16-digit number that's all over the pace, so you have a way to go.

The other part of the assessment of password hashing is to consider alternative mitigations. A bank isn't some service hosted on a $5 VPS, it can afford better security. For example, banks tend to have very tight controls on their employees, which reduces the risk of an insider breach. Against outsiders, storing the passwords in an HSM is a good but awkward defense (in particular, backups are hard). Encrypting the passwords and storing the encryption keys split over several smart cards is another possible mitigation — it doesn't protect the live system but it's effective for backups. Encryption is a very good mitigation of threat #1 (it works even against extremely weak passwords) provided that the key and the password database are sufficiently independent so that they are hard to steal together.

All things considered, password hashing is not a panacea in this scenario. While it isn't a bad idea, a qualitative analysis shows that an alternative approach that requires a reversible password storage may provide better security. It would take a very thorough study of fraud and usability to assess quantatively whether using random characters from passwords for authentication is worth foregoing password hashing.

Gilles 'SO- stop being evil'
  • 51,415
  • 13
  • 121
  • 180
  • Do you know of any analysis showing it to be of benefit? Since it goes against more or less standard practice, I would suggest that the burden of proof is to show that it is of enough benefit to risk a central compromise to all accounts simultaneously. I dare say a single point of catastrophic failure is almost certainly always very bad, no matter how unlikely you can make that single point of failure from occurring. – AJ Henderson May 31 '13 at 03:49
  • @AJHenderson dont forget, no one is saying tha passwords are - or should be - stored in plain text, these should still be encrypted using reversible encryption. It is still FAR FAR FAR from trivial, or even feasible, to expect to massively decrypt all of them at once, or even just a few. The main difference in risk is access to the encryption key, which Gilles related to. – AviD May 31 '13 at 06:30
  • @AJHenderson As I explain, the threat model for a bank is a bit different from your average service provider, so standard practice can be different. Unfortunately I doubt that you'll find precise enough studies publicly. Do note that storing passwords in an HSM is superior to hashing, because it means the attacker has to do online guesses — no matter how slow your hash is, it isn't going to protect a 4-digit PIN. – Gilles 'SO- stop being evil' May 31 '13 at 08:05
  • @AviD But if the database can be compromised, it's not so much of a stretch for any software decryption logic/keys (ignoring HSM) to be not far behind? It's a possibility rather than "unthinkable huge computation effort"? – Pricey May 31 '13 at 11:23
  • @PriceChild The idea is to split the database and the key as much as possible. They both have to be present on the live system, but at least they should be backed up in different places (and the key should only be present in RAM, on smartcards and on paper, never on a hard disk or tape). An HSM that stores the database is even better as it forces the password brute-forcing to be online (assuming the HSM isn't breached). – Gilles 'SO- stop being evil' May 31 '13 at 11:26
  • @Gilles Really like your answer. It's a really great idea about how the password scheme mitigates a different risk to the one I was initially thinking about. I may have made my edit to that last comment too late though, in my comment I was wanting to ignore schemes that use an HSM. "If you use HSM" could probably be a different, equally valid answer? – Pricey May 31 '13 at 11:28
  • @Gilles - are there any HSMs that support the select character approach? I agree that an HSM with full verify only would be secure, but by very nature, a pick 2 or even pick 4 would be by nature in secure as the HSM's verify operations give away far more about the password than a full verification, even if it is somehow natively supported without having to decrypt the password. – AJ Henderson May 31 '13 at 12:54
  • @AJHenderson [Depends on your choice of HSM](http://security.stackexchange.com/questions/36664/criteria-for-selecting-an-hsm/36673#36673). Even if you verify character by character, the HSM can limit the rate enough that there would be a very low expected number of cracked passwords until the HSM's theft is detected and the accounts are locked, or the attacker triggers the HSM's intrusion alarm. – Gilles 'SO- stop being evil' May 31 '13 at 13:07
  • @Gilles - how can it limit the rate that much against distributed attacking though. I'm sure the rate of logins on a large bank's system must be pretty high, so I'd expect that they wouldn't be able to use such a feature particularly effectively. That said, a fixed counter on individual keys could work to provide lock out with reasonably acceptable security. – AJ Henderson May 31 '13 at 13:12
  • @AJHenderson Yes, of course you'd need a counter per password. This way the attacker would have to make many attempts on distinct accounts, each with a very low chance of success. And that's a detectable pattern too: if 99% of the authentication attempts are failing, there's an attack going on. – Gilles 'SO- stop being evil' May 31 '13 at 13:14
2

My question is whether this can be justified?

Sure.

Hashing passwords is not a security fundamental. It is a very specific mitigation for the case where your database gets compromised. Even in that case all it does is give you a time buffer for you to detect and analyse the compromise, communicate to users and allow them to change passwords.

A database leak is still a disastrous case for most systems, but at least with strong hashing other systems where the user has re-used the password are less likely to get hurt by your disaster.

So you can certainly justify encrypted-but-recoverable passwords if you have alternative measures that make that database leak less likely or less damaging.

That could include splitting of the sensitive data/encryption keys so that multiple systems have to be compromised to recover the passwords; putting password checking on its own server with locked down access and strong auditing against unauthorised use and detection of unusual volumes that might indicate a compromised front-end server; password format rules that make re-use on other systems unlikely... and so on.

Is this really a better idea?

There isn't an easy answer to that.

The benefit isn't readily quantifiable at the moment as there's not much accepted data out there to analyse, and benefits would certainly depend on other aspects of an organisation's security profile. (Plus it's easy to make mistakes in implementation of partial password that can scupper the potential benefits, and usually partial passwords are combined with some other form of credential as on they're own they are susceptible to brute force.)

Similarly the cost to the organisation is dependent on what other factors are employed to deter or mitigate the database leak case, and how well they fit within existing practices.

To summarise: It Depends. :-)

bobince
  • 12,534
  • 1
  • 27
  • 42
  • My question is whether there is any reason not to implement this "very specific mitigation". If there isn't then surely your answer should be "no"? If there is a reason to hold non-hashed passwords then what is it? – Pricey May 31 '13 at 11:22
  • 1
    @PriceChild: self-evidently from your question there is a good reason not to implement the mitigation of password hashing: because it's not compatible with the competing security measure of partial-password. If you think the phishing-defence value of partial-password is worth more than the database-leak-mitigation value of password-hashing, then you don't implement password hashing (and instead go with some manner of recoverable encryption). Obviously. – bobince May 31 '13 at 13:05
  • "Self-evidently", if you think that's an answer, make it an/your answer? Gilles's answer explains a reason for partial passwords (not what I assumed) that makes sense. It's an answer like that which is why I asked this question. "Obviously". – Pricey May 31 '13 at 13:28
0

It can be justified, but it isn't necessarily better.

That said, imagine a service that log's into another service on your behalf--for example: Mint.com. It logs into your bank account and pulls your data.

Unless your bank implements OAuth, or another delegation service, Mint.com will have to store your bank credentials encrypted, so that it can act on your behalf. Prior to Twitter implementing OAuth, this was how legacy twitter apps worked with your twitter account.

This notion of storing someone's passwords on your system is a security smell, which gave rise to the trust delegation that occurs with an OAuth type system.

Also, making a generalized statement such as "hashing is better than encryption", in a vacuum, wrt security doesn't make a lot of sense given how nuanced security actually is. A poor implementation of password hashing (for example using MD5) with no-salt can be worse for your systems security than storing customer's passwords encrypted, so "better" is subjective.

Alan
  • 117
  • 4
  • 2
    "weak salt/hash solution is worse than a good symmetric cipher" - I would argue that if you can 'easily' obtain the plaintext from both then they are equally as bad? – Pricey May 30 '13 at 19:44
  • In the kind of setup you describe, the passphrase for the encrypted credentials should not be stored on the server. It should require the user to login to be able to make use of them. Storing a password for first party authentication using a symmetric key is always bad and third party is bad if it is recoverable without a first party login. – AJ Henderson May 30 '13 at 20:00
  • 1
    @PriceChild It's a fallacy to assume that symmetric encryption equals "easily obtainable plain-text." If that were necessarily true, it wouldn't be useful for **any** persistent data storage, which is demonstrably false. – Xander May 30 '13 at 22:15
  • @Xander Yes, not the greatest of wording. My intention was that if you have reversible encryption, there's one more attack vector in finding the key. It's something that isn't there for a good hash. – Pricey May 30 '13 at 22:35
  • http://security.stackexchange.com/questions/4830/how-do-some-sites-e-g-online-banks-only-ask-for-specific-characters-from-a-pa – Lucas Kauffman May 30 '13 at 22:40
  • 1
    @PriceChild I agree that's true, however, if you don't get the key, you can't get any of the passwords, which isn't true even with a *good* hash algorithm and an implementation that includes salt. There are always trade-offs, so I simply suggest that the idea that passwords should *always* be salted and hashed, without exception, is cargo cult security rather than real risk management. – Xander May 30 '13 at 23:23
  • 1
    @AJHenderson: Your comment isn't valid within the context of the question and answer. Question: Can storing encrypted passwords be justified. Yes. When acting on behalf of a third party. If the key is my password to the 3rd party service, is irrelevant (within the context of the question). – Alan May 31 '13 at 00:20
  • @Alan - your answer makes it relevant as you specifically give a situation where the password should only be stored protected by your mint.com password. This is not authentication in the way the question was talking about. – AJ Henderson May 31 '13 at 02:45
  • 1
    Though I suppose in fairness, it is a password being stored without a hash even if it isn't being stored in a centrally recoverable manner. – AJ Henderson May 31 '13 at 03:52
  • The bigger problem I had with the answer was the line "A weak salt/hash solution is worse than a good symmetric cipher, so "better" is relative." That seems off. If the server is compromised, a good symmetric cipher offers no protection unless the key manages to stay safe. Even then, encryption of the hash is better still. – AJ Henderson May 31 '13 at 03:53
  • That line was specific to whether "hashing vs encryption" is better. That question is slightly nuanced. If you are talking about properly implementing password hashing, then yes, it's better. However, it's very easy to screw the implementation of password hashing up, so I hold that you can't make a blanket statement saying that password-hashing is always better than encryption. – Alan May 31 '13 at 04:05
  • @Alan - ah, ok, I get what you are saying now. A well implemented symmetric encryption/HSM system would be more secure than a weak hash with no salting. If you clarify that in your answer, I think you might find that several of the down votes go away. (Mine will.) – AJ Henderson May 31 '13 at 22:26
  • @Xander I understand and agree with your fist sentence. The second though I've no idea what you're talking about? If you do anything badly then of course it'll go badly? – Pricey May 31 '13 at 22:35
  • @PriceChild You're trying to dumb-down security to an axiom that simply isn't true. To say that you should "always salt and hash passwords" without thought or question is just wrong. – Xander May 31 '13 at 22:39
  • @Xander Where've I said that? The entire point of me asking this question in the first place was to understand why it isn't required. – Pricey May 31 '13 at 23:51
  • @PriceChild Right, and "you" was probably the wrong word. I wasn't calling you out personally, I was re-stating the second sentence of my comment, which you had a question about. It's the very briefest explanation for why the answer to the question you asked "Is there any good reason not to salt/hash passwords you store?" is yes. There is no single universal implementation for secure password storage. Gilles and bobince both added far more detailed answers. – Xander Jun 01 '13 at 00:12
-1

While it is possible that the password could be stored with an encrypted HSM that restricts attempts to verify the data presented with a high degree of security, I still think the claims of what security are added are dubious compared to the information leakage presented by checking only a subset of a password. Since the gains seem dubious to me, I'd say there is no good reason to store a password in a recoverable format for authentication purposes. There is also no good way to verify if proper HSM procedures are being followed as we've seen boneheaded security decisions by large companies (and even major banks) in the past. If an HSM isn't being properly used it is a huge risk as any compromise in the database will result in leaking of all the account credentials. Passwords for authentication should always be stored in a one-way format to prevent recovery by attackers.

Note that if weak hashes are a concern, you could also always encrypt the hash to get the best of both worlds since storing it recoverable is critically dependent on key protection. Because of this, even if you have "reliable" key protection, hashing still offers more security for preventing a single point of failure.

When a password needs to be stored for use with a third party service, the password should be encrypted symmetrically with a key derived from the user's password. In this way, the passwords that are stored are unrecoverable in the event of a database breach since the stored passwords are only accessible when a password (which only the user has) is entered.

AJ Henderson
  • 41,896
  • 5
  • 63
  • 110
  • If passwords are encrypted in the database, both the data *and* the key must be leaked. Poorly hashed passwords are potentially far more vulnerable, so I think there are a number of unstated assumptions that must be clarified before this answer is reasonably valid. – Xander May 30 '13 at 22:29
  • Sorry I down voted this one. This concept is employed by different banks and the security and logic behind it is valid – Lucas Kauffman May 30 '13 at 22:36
  • 1
    HSBC, Natwest and the Verified by Visa program all do this (or variations on it). I suspect many other banks do too. It might be quite difficult to find a bank that doesn't. (Probably worth noting that many banks offer two-factor authentication along with the parts-of-passwords pattern). – Ladadadada May 30 '13 at 22:36
  • http://security.stackexchange.com/questions/4830/how-do-some-sites-e-g-online-banks-only-ask-for-specific-characters-from-a-pa – Lucas Kauffman May 30 '13 at 22:39
  • 2
    @Ladadadada - HSBC are the same people that send out e-mails telling people to use their real usernames and passwords at myfraudprevention.com. I don't exactly take them very seriously for evidence that something is a good idea. – AJ Henderson May 31 '13 at 02:49
  • @Xander - in that case encrypt the hash and you get both protections. – AJ Henderson May 31 '13 at 02:50
  • @LucasKauffman - I remember that question but I'm still not sold on the value of such a system. It wouldn't take long for an average password to be compromised if they could collect the characters being entered. It also seems dubious to me that giving away more information about a smaller subset of the characters doesn't make it easier to guess the password outright. – AJ Henderson May 31 '13 at 02:56
  • The main argument that I've heard is to prevent phishing, but then a phishing site simply has to ask for the full password to "verify their connection" and the majority of users will give the full password rendering the protection utterly useless. – AJ Henderson May 31 '13 at 03:04
  • @Alan - I am attempting to refute a suggestion as to if there is a good reason. My argument is that it does not in fact add any measurable security to preventing phishing and therefore isn't phishing protection isn't a valid argument. – AJ Henderson May 31 '13 at 03:39
  • @LucasKauffman - an interesting idea that occurred to me while in the shower is that an alternate approach could avoid centralizing all passwords while still allowing the partial password usage. On first login, a cookie is normally left to verify the computer as trusted for the account. The full password could be protected with an encryption key and returned in the cookie. This would distribute the risk. The server would only contain keys and the passwords would be recoverable for character validation when a trusted computer is used to connect. – AJ Henderson May 31 '13 at 03:40
  • @AJHenderson: Whoops. I failed at this internet game. I *meant* to post the above comment on my answer-comment thread... – Alan May 31 '13 at 03:44
  • @Alan - ah, that makes more sense. – AJ Henderson May 31 '13 at 03:48
  • @AJHenderson I think you may have meant a different website. However if a bank did ever email me my password, that would be enough to make me switch banks, even if they also used two-factor authentication. – Ladadadada May 31 '13 at 10:00
  • @Ladadadada - they didn't e-mail me the password. They e-mailed me saying my account might be compromised, please go and log in at myfraudprevention.com to verify my account. I immediately assumed it was a really bad phishing attempt, but then upon some investigation, I determined it actually WAS the bank after trying to login to check my account and being directed there by the bank's site and doing some additional searching because even then I couldn't believe they'd be so stupid as to train users to follow phishing attempts. – AJ Henderson May 31 '13 at 12:57
  • @AJHenderson are you sure about "hash then encrypt"? – Pricey May 31 '13 at 22:32
  • @PriceChild - I'm sure that as far as protecting a weak hash goes, encrypting the hash will provide equal protection to encrypting the password itself, but if the key is leaked, the hash still also has to be cracked. Simply pointing out that if you want the protection against weak passwords being cracked that is offered by symmetric encryption, it doesn't mean hashing couldn't be done as additional protection. I suppose you could also apply it in the opposite direction to ensure a more difficult to reverse hash. – AJ Henderson Jun 01 '13 at 00:50
-3

To ask for a part of a password is a mean to avoid it from "wearing out" too early (extreme case is OTP (One Time Password)).

I cannot be sure on how the password is stored, but it may be quite secure. The best I can think in 2 minutes is this (I am sure this can be improved):

  • salt+hash each part of the password, add some random data [to each part of the password] and encrypt the bundle with a symmetric key. Store the result.
  • when you get user's password piece, you decrypt the stored item, disregard the random data and compare the salt+hash

IMHO, this way of authentication may be ok as a part of the authentication process, but not the whole authentication process. If the authentication process to access my bank account is just "put letter 4 and 6 of your password" (and if your password is alphanumeric) I would be scared.

Borja Bolilla
  • 1
  • 1
  • 2
    Surely if you're only asking for 3 characters a time, you'll be hashing many 3-character words in the best case. Hashing is pretty much useless here because generating a list of all hashes is quick? – Pricey May 30 '13 at 20:52
  • Yes, although you can salt-hash the partial passwords it's pretty easy to brute force one triplet. From there you can use the two characters you know to brute-force the 'adjacent' triplets with shared characters and so guess the remaining pieces of password one character at a time. The level of protection offered by hashing partial passwords is so low it's almost never worth bothering. – bobince May 31 '13 at 09:42