15

When is it appropriate to use RFC2898DeriveBytes versus a typical hash?

Update

I now understand that a KDF is typically used to create a symmetric key for possible use in encrypting a stream. I also now understand that PBKDF2 obsoletes PasswordDeriveBytes. The intent of this question is to explore the possibility of re-purposing the KDF as a strong way to store a hashed password. The content I intend to encrypt is a Unicode string that a human can remember and type into a computer.

The reason I'm interested in the KDF is because it is

  1. Slow, and therefore computationally expensive to create a rainbow table
  2. The base library in .NET is FIPS approved and it's possible to use this KDF with NIST recommendations if we use SHA-256

Question

What are the arguments for/against using a KDF in this manner as opposed to the normal hashing method? (Disclaimer: What is the normal hashing method?)

Community
  • 1
makerofthings7
  • 50,488
  • 54
  • 253
  • 542
  • Note, from the answer given by @Thomas_Pornin, that this is not suitable for password storage. Generally you don't want to store passwords, you want to store something which only allows you to verify that the user has presented the right password - see [Password hashing - IT Security](http://security.stackexchange.com/questions/211/password-hashing). For other cases, see e.g. [Storing third-party auth info securely - IT Security](http://security.stackexchange.com/questions/1335/storing-third-party-auth-info-securely) – nealmcb Feb 12 '11 at 15:41
  • It's sad that the Microsoft documentation you refer to on PasswordDeriveBytes refers to "IETF RRC 2898". They mean [RFC 2898](http://tools.ietf.org/html/rfc2898), which describes more usage and security considerations. – nealmcb Feb 12 '11 at 15:41
  • Just to clarify the comments to date: PBKDF1 is out of date. It is superseded by the .NET crypto object named: [RFC2898DeriveBytes](http://msdn.microsoft.com/en-us/library/system.security.cryptography.rfc2898derivebytes.aspx) – makerofthings7 Feb 16 '11 at 22:23
  • I note the significantly changed question. The "FIPS approved" link to my answer in another question seems misleading. There I was only hinting at an argument that the primitives in PBKDF2, if adjusted to use SHA-256 rather than the default SHA-1, would match current NIST recommendations. But I've seen no FIPS specs on password hashing. My main point was that blowfish (bcrypt) is clearly not on NIST's list, while SHA-1 was and SHA-256 is now. – nealmcb Feb 18 '11 at 04:25
  • @nealmcb - Do the links look more accurate and clear? – makerofthings7 Feb 18 '11 at 07:28
  • Heh - so I have a vague recollection that RFC2898DeriveBytes allows the use of other hashes, but I don't do .NET. Have you confirmed that? Can you give us a snippet of how to use HMAC-SHA-256 with it? Also, is the library actually somehow FIPS certified, or is it just that it implements an algorithm that is approved? Finally, what parameters are you planning to use for salt length (64?) and iterations (1000? 4000?)? – nealmcb Feb 18 '11 at 15:30
  • related: [Does NIST really recommend PBKDF2 for password hashing?](http://security.stackexchange.com/questions/15065/does-nist-really-recommend-pbkdf2-for-password-hashing) – David Cary Jul 09 '12 at 17:27

6 Answers6

16

PasswordDeriveBytes implements the PBKDF1 key-derivation function. A KDF is a function which transform a piece of secret data (here, a "password", i.e. the kind of data which fits in a human brain and can be typed with human fingers) into a sequence of bits adequate for algorithms which need a symmetric key (e.g. symmetric encryption). A KDF is not meant for anything else, in particular password storage.

A hash function can be used as a KDF, provided that the symmetric key you need is no longer than the hash function output size. However, such a KDF would be very crude. One feature which good KDF should provide is to be adequately slow. This is because "passwords" are, by nature, vulnerable to exhaustive search (also known as "dictionary attack": users tend to choose as passwords rather simple words or combinations which can be guessed with only a few millions or billions tries). In a given system, one can usually tolerate a relatively slow KDF: a user trying to authenticate will not see the difference between a 1µs and a 1ms delay for the key-derivation function; but a 1000x slowdown is deadly for the attacker: it converts a one-day breaking effort into a three-years breaking effort.

PBKDF1 includes an "iteration count" which is a parameter meant exactly for that: to make the key derivation adequately slow, in a configurable way. A simple hash function is just too fast for that. Usage as a KDF is precisely where you would prefer PBKDF1 over a hash function. Actually, PBKDF1 is not recommended; PBKDF2, from the same standard, is supposed to be more robust.

Hash functions are much more generic objects than KDF, and have many other usages, which KDF do not fulfill.

What you want to do is unclear: you use the term "signature", which normally means "asymmetric digital signature" as with RSA or ECDSA; there are some people who tend to use the term "signature" to designate a symmetric integrity check, such as a MAC (calling it a "signature" is improper, but widespread). However, this entails some piece of secret at some point, a key, and a hash function is key-less.

Community
  • 1
Thomas Pornin
  • 322,884
  • 58
  • 787
  • 955
  • +1 for explaining the details *much* better than I did. – AviD Feb 08 '11 at 18:38
  • Thanks for the great answer. Also I found this link on MSDN that provides even more detail – makerofthings7 Feb 08 '11 at 23:50
  • .. link to MSDN: http://blogs.msdn.com/b/shawnfa/archive/2004/04/14/113514.aspx – makerofthings7 Feb 09 '11 at 00:04
  • @Thomas What is the correct way to store a password if not PBKDF2? – makerofthings7 Feb 11 '11 at 19:05
  • Link: How *should* one encrypt a password if not by a KDF: http://security.stackexchange.com/questions/2131/reference-implementation-of-a-c-hash-for-secure-password-storage – makerofthings7 Feb 13 '11 at 01:10
  • 1
    @Thomas RFC2898 says PBKDF1 can be used for password checking, [`where the output of the key derivation function is stored (along with the salt and iteration count) for the purposes of subsequent verification of a password`](http://www.ietf.org/rfc/rfc2898.txt) Can you please clarify your answer? – makerofthings7 Feb 13 '11 at 17:11
  • 2
    For password storage you want a one-way function with a fixed, large enough output. For key derivation you want a collision-free function with a configurable output size. It is conceivably possible to make a KDF which _also_ has security characteristics good enough for password storage (assuming you are using a proper, fixed output size) but this is not automatic. PBKDF2 has been published as a KDF and has been analyzed as such. Whether it can be subverted into a password storage mechanism remains to be proven. – Thomas Pornin Feb 14 '11 at 13:36
  • 1
    @makerofthings, your quote is out of context. In fact RFC2898 specifically said, 10 years ago, that PBKDF1 is not recommended for any new applications. Based on @Thomas' comment I'd say you might want to at least use PBKDF2 which can produce longer output. – nealmcb Feb 15 '11 at 02:52
  • Thank you. Does that mean that PBKDF2 creates an output that is better than any other such as a SHA 256 hash? (for the purpose of password storage and later comparison)? – makerofthings7 Feb 16 '11 at 23:08
  • @makerofthings, as @Thomas explains, PBKDF2 is designed and intended for key derivation, rather than for password storage. Perhaps it would help to clarify the original question, or post a new question, in a way that explains for what purpose you want this cryptographic primitive? – D.W. Feb 17 '11 at 01:35
  • @D.W. - Question clarified. Thanks for the advice. – makerofthings7 Feb 18 '11 at 02:57
  • 1
    @D.W. @Thomas But note the section quoted by @makerofthings in which the RFC anticipates using the functions for storing password verifiers. I also don't understand why the length of even PBKDF1 might be inadequate. Longer hashes make for slightly longer rainbow tables, but that doesn't matter given the default of 64-bit salt protection, right? – nealmcb Feb 18 '11 at 04:20
  • 1
    @D.W. @Thomas @nealmcb fyi - Found another guru who recommends using PBKDF2 to hash a password: http://www.daemonology.net/blog/2009-06-11-cryptographic-right-answers.html – makerofthings7 Apr 07 '11 at 21:05
14

The NIST approves of PBKDF2 when hashing and storing passwords, however that is not its original intended purpose. Notably, StackExchange also uses PBKDF2 for the same purpose. Source code is available here.

See this answer for a comparison between BCrypt and PBKDF2. BCrypt is the the more conventional method of storing passwords.

I'm considering PBKDF2 since it's built into .NET and is already FIPS compliant.

Community
  • 1
makerofthings7
  • 50,488
  • 54
  • 253
  • 542
  • 2
    I disagree on the "nist-sp800-132" approves PBKDF2 for "hashing and storing passwords". It actually talks about using PBKDF2 as a KDF for deriving data protection keys which are used to encrypt stored data. – eckes Nov 22 '12 at 19:58
6

Unless I've missed something, PasswordDeriveBytes - and other PBKDF implementations - are not intended for storing passwords, nor are they to be used instead of a "typical" hash.

What it is intended for, is to create an encryption key, for symmetric encryption, based on a user-provided password.

To clarify, consider the following situation:
You have an application, that requires encrypting a file. This is done at the user's discretion, and can be decrypted at his choice.
Oh, let's say MS Word has a feature to encrypt it's contents. Or an encrypted ZIP file.
You want to give the user the control to access it, but don't want to rely on the user to generate a strong, random, exactly-256-bits-key, which would make it difficult to remember, etc.
So, you allow the user to set any password he wants - and derive the encryption key from that.

At no point are you storing the password, or using a simple hash to store it. It is only intended for creating key material.

AviD
  • 72,708
  • 22
  • 137
  • 218
4

PBKDF2 is better than PBKDF1, which was deprecated in RFC 2898 10 years ago. It is implemented for .NET in Rfc2898DeriveBytes Class (System.Security.Cryptography)

I'm glad to also see that as of Java 6 there is an implementation of PBKDF2: PBKDF2WithHmacSHA1 in SunJCE.

For earlier versions of Java, as discussed at BlackBerry App Security - Stack Overflow, an implementation of PBKDF2 is at A free Java implementation of RFC 2898 / PKCS#5 PBKDF2

I wonder, though, if using SHA-256 would be better than SHA-1, which is now deprecated.

Community
  • 1
nealmcb
  • 20,693
  • 6
  • 71
  • 117
3

Using RFC2898DeriveBytes with a non trivial iteration count should be better than using a straight hash function for authentication purposes.

First it requires you to put in a salt, and while it's still possible for the developer to do something stupid like hard code the salt, it's at least one step closer than where developers normally messup which is not salt and all. Just having a salt makes its less likely a rainbow table already out there is going to work on someones trivial password and if you salt randomly for each user it stops leakage of information when multiple users have the same password and makes those easy passwords very hard to get because you'd have to generate table for each salt.

2nd while RFC2898DeriveBytes unfortunately doesn't let you choose the hash function and it uses HMAC-SHA1 and it's not really a problem as the nice thing about it versus PBKDF1 is then size of your key can be as large as you practically could want. So that helps with making it difficult to store and disseminate rainbow tables vs any standard hash algorithm.

Finally, the iteration counts are key! Have a really high count slow that hash calculation down, standard hash algorithms are designed to be fast, data storage is growing exponentially you can't just count on just the size of your key.

jbtule
  • 149
  • 5
1

The OP's question and the goal of said question are somewhat contradictory. As everyone is pointing out, PBKDF2 is not primarily used for password enc/storage, but key generation. Most commonly, it is used in WPA2 PMK and other WPA key generation schemes, using such example components as the password + a salt of the SSID name and SSID name length, which is then run through a 4096 iteration SHA1 hashing.

When encrypting passes, the most important things to require for security sake are length and a salt that is dynamic and not transparent to the user (ie: the exploiter cannot determine what the salt is). If you use an AES encrypted store with a minimum 20 character password, a random salt, and 4K+ iterations of SHA-256 (PBKDF2 recommends at least 1K iterations of SHA1 for its purposes), a hacker is going to have a very difficult time breaking that password (see: order of magnitude in years or decades if not longer).

Password encryption is also highly dependent on end use (speed, necessity of security level, etc), so without knowing what potential end use is it's more difficult to recommend an ideal solution.

mrnap
  • 1,308
  • 9
  • 15
  • 1
    For most salted encryption schemes which are used out there in the wild the salt is not hidden. Thats not a helpfull restriction. You cannot legitimately re-produce the hashing if you dont know the salt. – eckes Nov 22 '12 at 20:00