8

I've seen a lot of questions and answers acknowledging the dangers of inventing your own crypto algorithms — and even of implementing proven algorithms yourself. And, I don't disagree. I might even argue most of us don't even know how to safely use the existing libraries!

But, who should do it? What education, experience, or environment warrants writing your own crypto?

We dogmatically tell everyone who asks "don't roll your own! Use an existing library" But, somebody writes cryptographically secure algorithms and code. Who?

Who wouldn't we scold for "not using an existing library"? (And why do we trust them?)

As an aside, is there any place for the "common" developer to learn and contribute in a safe and beneficial or non-destructive manner?

svidgen
  • 711
  • 5
  • 13
  • 1
    Anyone can do it.But it should only be believed in if it has been peer reviewed and tried to be broken by many other security professionals. – yeah_well May 07 '19 at 17:56
  • 2
    FWIW, I think it's a fine question, and couldn't find an adequate duplicate when I looked (I though I remembered one either here or on Crypto.) – Xander May 07 '19 at 20:06
  • @Xander I seem to remember something that I think we ended up migrating to crypto. It was a couple of years ago maybe? – schroeder May 07 '19 at 21:04
  • @schroeder Could have been. That's jives with the time frame I had in mind, but I couldn't find it. – Xander May 07 '19 at 22:16
  • Why does every post of this nature assume that there *is* a library. Who is writing the library ? – mckenzm May 08 '19 at 03:50
  • 1
    @mckenzm every language has a crypto library – schroeder May 08 '19 at 07:01

5 Answers5

9

You noticed the unfulfillable requirement: you are never told that you are ready to write crypto code.

I think the reason the "never roll your own" recommendation is always advised is because one is seeking recommendations. As a lower bar, the point where you no longer rely on public opinion for advice is when you can start thinking about crypto.

And, at that point, you fully assume all responsibility for what you create. The social group of developers never told you you could do it. You just decided to do it. And you live with the consequences.

Now, more specific than that, several experts recommend first trying to break other's crypto libraries before trying to write your own. Once you can break what's out there, you are more likely to be aware of all of the nuanced pitfalls that come with writing your own library.

But myself, I think of it with a martial arts metaphor. Think of an art where the black belt test is notorious. It shreds egos like it was As Seen On TV. You only get to take the test once. So many ask "am I ready for the test?" And the answer is invariably "You are never ready for the test."

So when are you ready for the test? When you realize you don't need a test to let people know you're good enough to be a black belt. You know it. So you just put on a black belt, stare down the experts who were supposed to give you the test, and they nod as if to say "yep, that was the trick the whole time."

Cort Ammon
  • 9,216
  • 3
  • 26
  • 26
  • 1
    +1 for the final paragraph. That is a _perfect_ way to explain when you are ready. – forest May 09 '19 at 02:03
  • This is the most perfect description that I've ever heard. I need to use this. It's not a patronizing dismissal like most of the other answers, it's more of a zen statement of responsibility. When you build it, you own it. – Stack Tracer May 09 '19 at 03:45
  • 2
    I really like this answer. But, I'm surprised it's so upvoted: it leaves a fair amount of room for one to be afflicted by the Dunning-Kruger effect without any concrete guidance. I expect a real novice won't even be able to recognize who the "experts" are and who to stare down to prove themselves. The consultants we hire from a general engineering shop aren't crypto experts; but, they often operate as though they are. Would I have known 15 years ago they're not crypto masters? (*Probably not.*) Would I have come up with my own crazy schemes to secure company assets? ... I certainly might have. – svidgen May 10 '19 at 15:40
  • That said ... I'll re-read a few more times over a few days and think on it. Maybe a few things need to be clarified in the way they're phrased here. (Or maybe not...) – svidgen May 10 '19 at 15:43
  • 1
    Agreed about the Dunning-Kruger effect. It was a major concern when typing the answer. I added two small bits to try to deal with that. The first is that you assume responsibility when you go down this path. Are you *actually* ready to be responsible for, say, $100M worth of assets? Does your boss think you're ready? And the second is that, when you are ready, you don't just put the black belt on. You stare down the experts afterwards. If they're all shaking their head, that's the best evidence you'll ever get to suggest you made a mistake. If you don't know who the experts are... – Cort Ammon May 10 '19 at 15:53
  • 2
    ... then you most certainly have made a grave tactical error. But if they look at you and nod, then you *still* don't know if you're falling victim to Dunning-Kruger, but you've got the best evidence you're ever going to have that you aren't. – Cort Ammon May 10 '19 at 15:54
  • 3
    A classic example I can think of is Ponic, which was submitted to the NIST hash challenge which produced SHA-3. Ponic was put together by a 15 year old who thought he could make it happen. His submission was "rejected" meaning NIST discarded it without even taking the time to put together a paper explaining why. Later, it was demonstrated that there were 2nd preimage attacks that showed glaring weaknesses. But for all that, you'll note that the kid didn't go around telling everyone "I made an awesome hash because I can do cryptography. Everybody should use it." – Cort Ammon May 10 '19 at 16:01
  • 4
    No. He went straight to the experts and stared them down. Yes, he got ripped to shreds, but he did indeed have what it took to stare them in the eye. I haven't tracked his career since then, but I hope he stuck with it and we see his name on papers in the future! – Cort Ammon May 10 '19 at 16:01
  • 1
    @CortAmmon The 2nd preimage against Ponic is not practical as it requires 2^265 operations to find a 512-bit preimage. It's still a severe attack but it's not horrible. Of course, attacks only get better... – forest May 30 '19 at 03:06
4

If you have to ask this question, you shouldn't write crypto code.

If you're talking about implementing an existing algorithm, all it takes is an experienced programmer with an understanding of side-channel attacks, secure program design, and a good understanding of cryptography. It's not easy, but it's not a high bar. These people can be hired to write the cryptographic libraries that everyone else should be pushed to use.

If you're talking about developing an entire new cryptographic algorithm, on the other hand, the answer is no one. Yes, really. No one. Not even the best cryptographer should be writing their own algorithm and using it without it undergoing serious review by other cryptographers. However, they are the people whose algorithms are most likely to withstand cryptanalysis. If you have a PhD in cryptography and have a well-known name in those circles, then you might be able to either assist with, or design an algorithm that has more than a snowball's chance in hell to withstand the barrage of attacks other cryptographers will subject your design to, assuming you can convince them that it's worth it.

forest
  • 65,613
  • 20
  • 208
  • 262
2

First, note that I'm talking about implementing actual cryptography functions, i.e. encryption, signatures etc. This is different from just using existing cryptography functions (much easier, but still many errors possible) and different from designing cryptographic algorithms (much harder).

Also, while I write code in a security context and also use existing cryptographic functions I don't write crypto code. But I understand that many things can go wrong when writing such code because I've read enough examples about cases where things went wrong. And I also know that I don't understand enough of the topic yet to write secure crypto code. Specifically I know that it is not obvious even for most experienced programmers when crypto code behaves wrong in some edge-cases or has unintended side-effects which might leak parts of secrets.

Given that I better don't write such code myself it is hard to give good recommendation on who should write such code, but I try to extrapolate from my experience with writing non-crypto software in a security context.
So who should write crypto code then - at least crypto code which gets used in production?

If you have the feeling that you'll understood all of the problems with writing such code then you probably overestimated your knowledge - see Dunning–Kruger effect. If you have some coworkers who already have a more widely acknowledged reputation for writing good crypto code let them review your code. If they find nothing to complain you should better be suspicious since it is unlikely that your code does not have even a tiny risk which should at least be discussed. Thus you should better get another opinion.

After doing this for a while you probably get the necessary experience to write good crypto code. But even at this stage you likely feel more comfortable if you ask coworkers for review and discuss potential risks with them, even if these seem mostly theoretical to you.

Steffen Ullrich
  • 190,458
  • 29
  • 381
  • 434
  • 1
    To be clear, I have zero intention of rolling any of my own crypto for any production purpose whatsoever. I'm aware that this stuff is a thousand percent beyond me. But, given that we dogmatically just tell *everyone* "don't roll your own crypto" ... I'm curious as to who *wouldn't* be told to "just use an existing library" ... you know? – svidgen May 07 '19 at 19:49
  • @svidgen: In this context, the term "everyone" means "almost everyone." As you have pointed out, it does not make logical sense for "everyone" to literally mean everyone. – hft May 07 '19 at 20:35
  • 1
    @hft Exactly. So ... who *don't* you tell "don't roll your own" -- and *why*. – svidgen May 07 '19 at 20:39
  • @svidgen: Everybody who comes with an idea of implementing some fancy new algorithm should show that this new idea tackles a problem where no usable solution exist yet and how its design is based on existing ideas which are successfully used in practice. Most of the cases I've seen so far are simply that users were unaware of existing solutions and instead of asking for help to find a solution they quickly designed their own one and asked for feedback. This actually reflects what I've wrote about Dunning-Kruger effect, i.e. these users believed that they properly understood the complexity. – Steffen Ullrich May 07 '19 at 20:55
  • @svidgen: In other words: if somebody shows serious research so that one can see that he really knows what he is talking about I would not say *"don't roll your own"*. Instead I would point out that this is likely the wrong place to ask and that he better should ask at [crypto.se]. If the community over there still thinks that there is no better solution than the proposed one (i.e. provides actual positiv feedback from obvious experts instead of just ignoring the question), then the idea likely wasn't that bad. – Steffen Ullrich May 07 '19 at 21:04
  • Geeze ... I totally forgot Crypto.SE exists. That's probably where the more detailed, nuanced answer I'm looking for would come from. Right? ... I mean, honestly, given that we have a dedicated crypto site here, your last comment may sufficiently answer at the right level for *this site*. – svidgen May 07 '19 at 21:12
  • @svidgen: I'm not sure if this would be a better place to ask but you might try. For me they seem to be focused more in the specific algorithms and less on how to develop secure software in more general. – Steffen Ullrich May 07 '19 at 21:25
  • 2
    With regard to this: "*using* existing cryptography functions (much easier, but still *many errors possible*)" - this begs yet another question: who "SHOULD" even _USE_ crypto libraries? – The_Sympathizer May 08 '19 at 04:56
  • 1
    @The_Sympathizer For high-level crypto libraries like NaCl, anyone who has a solid understanding of the language they are programming in and at least a basic understanding of cryptography can. For low-level crypto libraries like OpenSSL, you need to have a much better understanding of the library. – forest May 09 '19 at 02:09
2

Schneier's Law is relevant here: Any person can devise a code that he or she cannot imagine being breakable; the value comes when a bunch of other people try to break it and cannot.

According to that logic, we should never allow any given person to write his or her own crypto for use. But we can allow any person to write one, then put it out for review, then improve it, and if the community eventually agrees that it's a pretty good system, then it's okay for use. Don't trust a person; trust a community and a process.

user3583489
  • 321
  • 1
  • 3
  • 1
    This looks like a big part the straightforward guidance I'm looking for. Who are the "others" who you'd subject your crypto algorithms and code to? And for how long before you'd consider your algorithm or code to be "safe to use?" – svidgen May 10 '19 at 15:48
1

The real question (as I see it) is not who can write it, it who gets to accept it as standard and actually use it. This is almost never a single person. But as has been stated this advice is not meant for large groups reviewing, refining, and attempting to break proposals. It was meant to stop a-guy-who-wants-some-crypto going all judge, jury and executioner and rolling their own, and accepting it themselves.

The acceptance thing isn't about the people that wrote it. You rarely need to eat your own dog food in cryptography. You try to convince your would-be users. How they decide is a different matter.

drjpizzle
  • 199
  • 4
  • Fair enough. So, who gets to set the standards? And why do we trust them? – svidgen May 13 '19 at 14:34
  • @svidgen That's a good question. I don't have a definitive answer to that. I also don't think there is one. However in most cases it's a major government (commonly the US) adopting one that set things in motion. After that its market forces. If its hugely well known about and considered as-of-yet unbroken its good enough for me. If others start to see it that way it gets used more, tested more, better supported etc. This perpetuates until it gets shown to be broken in some way. But in general why someone trusts A over B is down to them and not everyone is logical about it. – drjpizzle May 13 '19 at 17:07
  • Interesting that the US Goverment (or *any* government) would be at the top of the trust list ... – svidgen May 13 '19 at 17:54
  • @svidgen I don't think that's really how it is, though yeah its an interesting setup. I think its more they're the ones that make the investment enough to commit. Plus they are a large group acting as unison. That doesn't happen too often. – drjpizzle May 14 '19 at 22:08