-2

By now, everyone on here should know that rolling your own crypto is bad.

But what's about rolling your own (encryption supporting) network protocol?

I'm working on an open-source project (by now it's an idea to be honest) where I need to exchange a lot of binary data between clients, but transfering text information with the data or even alone is an important aspect of inter-node communication and having two seperate protocols for doing so is an outright tdwtf.

My basic idea would be to write my own protocol specification with header, text- and bin-section and encrypt all of this with RSA (the question is not about reviewing this idea).

Is there anything bad in writing my own protocol security-wise* instead of relying on something existing, yet maybe not fitting?

* I do see that this maybe will make it harder for third-party clients and that this is not a security boundary.

EDIT: To clarify, I'm talking about a protocol on top of TCP (like HTTP). The question is about the security implications when I'm rolling my own protocol with a well known cipher integrated.

Sebb
  • 733
  • 1
  • 5
  • 11
  • I'm talking about a protocol like HTTP, so on the TCP layer. The linked questions are about rolling your own crypto, which is making your own way of encrypting things. I'm asking about making a own protocol like http, which would best fit my programs needs and the security implications of this (when using a known cipher). – Sebb Oct 02 '15 at 07:13
  • 1
    Have you looked at all the variants of TLS/SSL and found them to be not suitable for securing you custom protocol? – jhash Oct 02 '15 at 11:11

1 Answers1

2

Having pentested a number of custom network protocols (all implemented on top of TCP or UDP, sometimes with (D)TLS as well), I recommend against this idea. You will get it wrong, and then you'll have a remote exploit vector.

It is much, much safer to use an existing data format (JSON via HTTP is popular these days, though yes, that's a lot of overhead) with established server and client software, established parsing libraries for your data representation, and a fairly simple format (don't get fancy unless performance actually, sincerely, requires it). Use TLS (or DTLS if using UDP) to secure the traffic. Add your own authentication if required.

CBHacking
  • 42,359
  • 3
  • 76
  • 107