How to properly encrypt a communication channel between a client and a server (without SSL)?

Question

I have a poor knowledge in the world of cryptography, I'm just starting to learn it so I'm a newbie :)

As a project for getting into the field of cryptography I want to write a client\server program that can send and receive data in a secure manner (i.e encrypted).

Let's say I want to encrypt the traffic between a client and a server or between two clients. The service I'm running can only talk with the server or another client (which acts as server too) on a known port, the server basically listens for a connection on one side on a default port.

I want those services to be able to talk to each other with out the possibility that a third party who's listening on the network to be able to decrypt the content of the conversation or try to impersonate a service.

I'm not looking for a new way to exchange keys(or should I?), I can have the keys be saved securely in the services so when x sends a message to y, and back, they will be the only ones that can decrypt the content and response (off-course, only to a verified service).

• side note, the key can be saved in the program - hard coded or it can be generated from a combination of things (e.g IP, computer-name, user name or any other combination of constants) which can be passed to the client so it will be able to generate the key (lets say I'm passing the IP, computer name and a random seed, there will be a known formula to create the key from those values).

I need to use a symmetric key for the encryption, but maybe I can try and use two pre-generated keys as SSL does.

I know that if I'm using the same key again and again, a third party can sniff the traffic and get the key (mathematics at it's best, how is it done?), the more data he will collect, the easier it gets(?).

So, what should I do in order to promise that no-one will be able to get the encryption key (eventually reading the data)? Which cipher should I use? What else I need to know before I start my journey?

Answer 1

This is actually simpler than the others have suggested I think. Here is what I propose:

1. Generate a public/private key pair using RSA. You can do this on any unix machine using openssl: openssl genrsa -out rsa.private 2048
2. Distribute the public key as hardcoded in your client.
3. When the client logs in, the client generates a private shared key (also using openssl, or another well known encryption library) and encrypts it using the public key.
4. all data between client and server should be encrypted with the new shared key for the length of the session.

There is a lot of other stuff you can do to harden it, such as limiting sesison length to one or a few hours before re-generating keys, having the server sign message digests, etc. A full understanding of encryption will help you understand what those are and why, but for a basic implementation (for medium to low value data) this should be sufficient.

Answer 2

Before coding, read this: http://www.schneier.com/book-applied.html

This question is too broad to fit on StackExchange unfortunately. The length and breadth of cryptography is rarely traversed by a single individual with any degree of success without formal education in mathematics, computer science, and a smattering of engineering.

This book will help you learn common concepts, common mistakes, fark-ups to learn from, and examples to follow.

Answer 3

Basically you want the kind of tunnel that SSL provides. These things are actually difficult to get right.

SSL is an assembly of several things. There is the tunnel itself: once there is a shared secret (what is called the "pre-master secret" in the TLS specification), and client and server agree on which algorithms to use, then the individual data records can be encrypted, as is described in section 6 of the spec. To get to that point, SSL first performs the handshake in which the asymmetric cryptography occurs. If you control both client and server, and they already "inherently" share a secret, than you can skip the handshake.

During the handshake, the server sends an opaque blob (the "certificate") which the client may use to obtain the server's public key. If the client has an inherent knowledge of the server's public key, then it can totally ignore what the server sent, and simply use the known public key. Certificate validation (which is a rather complex piece of code) is used only when the client wants to discover dynamically the server's public key -- that's what occurs on the Web. But, depending on your scenario, this is not necessarily needed.

We now have on this very site a question about how SSL works with very descriptive answers (if lengthy).

If SSL does not inspire you, have a look at SSH. Same concepts, distinct protocol. SSH is described over several documents; begin with RFC 4251.

Answer 4

I realize that this is an old post, and hopefully I'm not wandering too much off topic, but a few thoughts.

First, in terms of pre-configuring the keys, it's a bad practice (especially if you're using an older encryption algorithm like DES where it's fairly straightforward to recover the key). You want to be able to rotate the keys periodically, not just use them in perpetuity; in principal at least the longer you use a key the less secure it becomes (because if someone were to get ahold of the key somehow they could then decrypt all communication that used that key).

In practice modern, properly-applied encryption algorithms are very difficult to crack and they're generally pretty resistant to known-plaintext and chosen-plaintext attacks but there are often other ways of getting at a key, such as hacking into - or gain physical access to - the server or client computer; this has actually happened to many companies recently (most notoriously Target). This particularly an issue if you're storing it on the server in plaintext. (There's a lot of documentation out there on how to store things like this properly - I've never tried this with a key but I imagine you'd want to follow procedures similar to how you'd secure a password).

The key-rotation issue wouldn't be a problem if you have ongoing physical access to both the client and the server, in which case you could manually update the keys periodically (as long as you didn't forget). (You'd still want to store them securely on both the client and the server in case someone hacked the server or client or got physical access to it).

As a general rule, keep in mind that security is always relative, so when you say you want it to be secure the question is always "how secure?" For example, I had a project where we were using affine ciphers to encrypt certain database values; those are easy to crack but our purpose in using it was to prevent access by casual observers, not deliberate malice, so it was sufficient. In practice, most older algorithms like this are already badly broken and consequently offer very little protection against a sufficiently-motivated attacker with adequate resources. For example, DES (Data Encryption Standard) is vulnerable to "brute force" attacks due to its keys being too short (even in the late '90's it was possible to brute force a key in under 24 hours) but it still offers at least some protection against unmotivated attackers.

Also, this probably isn't exactly what you meant in your original post but there are no practical algorithms where it's literally impossible for an attacker not to decrypt your message; for modern algorithms like AES it's impractical (but not, in principal, impossible) to do for now. (Key phrase: "For now." DES was secure when it was first developed but it was publically cracked 20 years later and it was probably possible for governments to do so long before that). There are some algorithms that have "perfect security," such as the one-time pad (where you have a fully random key that's as long as the message that you only use once); for these, it's not possible to discover any more about the message than its length without the key. However, this algorithm is impractical because of the key exchange problem; it's just as difficult to exchange the key securely as to exchange the message securely.

Answer 5

It can't be done without an independent distribution channel to transfer a key from the server to the client. For SSL, that's the whole certificate signing process and the public keys for certification authority root certificates that are included in web browsers. So your first problem is working out how to get a key from the server to the client in a way that can't be intercepted by a man-in-the-middle attack (i.e. not over the network).

Answer 6

I faced a similar situation in the past and solved using GET and POST requests over HTTPS. In Android it is rather simple to code and the included HTTP client takes care of all the crypto stuff for you, so you can have an easy solution and avoid the all-risky "brew your own crypto"

Answer 7

If you want to create a secure channel without SSL/TLS, then what you need to do is re-implement SSL/TLS, piece by piece. There isn't much (if anything) you can leave out and still be left with a secure connection -- everything you see there is the result of about 20 years of use and review and analysis.

This is a pretty signification part of the reason why you always hear that you shouldn't try to roll your own encryption. So rather than starting from scratch and making your own road, you're better off sitting down with the specs and learning how the existing implementations work.