8

I have been reviewing code for various internal apps that handle secure transmission of data with external systems. These follow the following process:

  • RSA encrypt data with 3rd party public key
  • RSA encrypt again using our own private key
  • Transmit data
  • 3rd party decrypts using our public key
  • 3rd party decrypts again using their own private key

In this system we generate new private/public keys for each third party and all public keys are distributed securely and then kept private.

I don't understand why we would encrypt data using our own private key. I suspect the designer's intent was to enable verification that the data originates from us and that same person had no knowledge of signing messages to implement that instead. Unfortunately, that person is no longer around to be asked.

I am happy that our data is acceptably safe because we encrypt that with the third party key, but private key encryption seems sufficiently non-standard that I feel some discomfort. Is there anything that I should be concerned about?

  • 1
    I don't think you can really answer this question without knowing the full details of the algorithm used. It is highly suspect, and I don't see why a signature could not be used. If it is because of message size then I would advice on using eliptic curves (first encryption, then a signature) instead of designing proprietary crypto algorithms/protocols. – Maarten Bodewes Feb 18 '12 at 00:44
  • I think you are on the correct train of thought. The only time I've heard of encrypting with a private key is so that whoever receives the data can know without a doubt it came from you –  Feb 16 '12 at 20:41
  • When encryption the actual data (as opposed to a correctly padded hash) with the private key, I don't think it even achieves that goal. – CodesInChaos Feb 16 '12 at 20:48

3 Answers3

9

It seems that the code you are looking at is taking a bit too literally the first description of how RSA signatures work.

When RSA was first described, it was an asymmetric encryption system: sender encrypts data with the recipient's public key, and the core operation is a modular exponentiation which takes as input a number modulo n (n is the modulus, a part of the public key) and returns a number modulo n. Then somebody said: hey, if we reverse the process and decrypt some input data (with a private key, necessarily), then we get a signature since anybody can perform the reverse operation (encrypt with the public key), which counts as signature verification.

This is an over-simplistic description of RSA signatures, and it does not actually work, because it neglects the effect of padding, which is necessary for security (see PKCS#1 for the Right Way to do RSA encryption and signatures). But the idea that signatures are encryption-in-reverse (or "encrypt with the private key") is unfortunately widespread, and lead to scores of homemade solutions which are all, invariably, weak in some way (but not necessarily in an obvious way). Chances are that you observe such a homemade solution.

Community
  • 1
Thomas Pornin
  • 322,884
  • 58
  • 787
  • 955
7

It's at least bad practice, and probably dangerous. I strongly recommend changing the scheme you use.

You shouldn't encrypt the content with your private key. You should sign it with your private key. While signing and encryption with private key are similar operations when using RSA, they should be considered separate. In particular signing works on a hash, not the actual content.

Depending on how exactly your current system is implemented, it might be possible for one third party to trick you into decrypting data received from another third party.

It might also be possible to trick you into signing something you didn't want to sign that way. But without the details of the scheme you use, I can't say for sure.

CodesInChaos
  • 11,964
  • 2
  • 40
  • 50
1

PKI algorithms generally use longer keys and thus are more computationally expensive. That's one reason things like SSL/TLS use PKI to setup the stream but then use a shared key cipher with a shorter key.

Whether it's risky to encrypt someone else's text with your key depends on the cipher being used. Some ciphers are vulnerable to chosen-plaintext attacks. Plain RSA is one of these. PKCS#1 isn't.

But as you say, if all you need is authentication, you can just sign it (thought that requires your private key as well.)

Steven Parkes
  • 119
  • 2
  • Encrypting data that's potentially controlled by an adversary with your private key sounds very dangerous to me. – CodesInChaos Feb 16 '12 at 20:49
  • @CodeInChaos: agreed. I missed the fact that the 3rd parties were the same. –  Feb 16 '12 at 21:08