0

I'm working with a program with a networking protocol that is using RSA keys to authenticate clients. I am looking to replace the custom authentication protocol with TLS (as it's a vetted protocol). I can use self-signed certificates or RFC 7250 for client authentication. The problem is that there are some use cases for the program where the standard PKI system (i.e. trusted root certificates) isn't viable for authenticating the server. (The program needs to be able to connect to servers on a LAN and it's not reasonable to have users setup their own roots to do it). Looking into "client-only authentication", I found questions like this one where the answer suggests reversing the TLS client and server roles. However, I'm not sure that's the right answer for my use case. I could also just have the server use its own cert/key as its identity and have the client accept any identity when it doesn't know what to expect. I have problems/questions about both solutions:

Using the server identities seems like the most sensible solution to me. Most connections will be able to use an authority to know what the server identity should be, the one case where an un-authenticated client needs to be able to connect will be simpler, and using the protocol in the correct direction might have some optimization advantages. The issue is that I don't know if client authentication is secure when the server authentication may not be. Looking at the specifications, the client and server authentication protocols are similar enough that it should still be secure, but since the spec doesn't allow for "client-only authentication", it's clearly not an intended use case. Is there any reason/evidence to believe that client authentication would or would not be secure when ignoring the server's authentication identity?

Having the client act as the TLS server, being a method suggested multiple times here, seems to be the method with the most guarunteed security, but would be a little tricker to implement. The main thing I'm interested in with this method is what other effects of switching the TLS roles would be. TLS is obviously an asymmetric protocol, but what other asymmetries are there other than authentication? The one I know of is that, while the server ultimately chooses the cipher suite, it's the client that indicates its cipher suite preference that the server normally considers. I found SSL_OP_CIPHER_SERVER_PREFERENCE to correct this in OpenSSL. What other effects of switching the TLS roles should I consider?

Which method would you reccomend in the end? To add a few points, I'm not worried about the server sometimes not being authenticated. Confidentiality isn't a high priority (just a side effect of authentication), and the protocol is mostly the client sending "commands" to the server, so someone impersonating a server isn't going to be able to do much. The only thing that server authentication could reasonably protect would be an optional password authentication, but I'm looking into implementing a PAKE for that, so it shouldn't be a concern.

Community
  • 1
LRFLEW
  • 103
  • 2

1 Answers1

2

Server authentication is done to protect against active man in the middle attacks by knowing that one is doing the key exchange directly with the expected server. Client authentication is done so that the server knows what client connected. It is best to stay within these roles when using the protocol, because these use cases are properly cared for by TLS.

This does not mean though that other cases of using TLS are inherently insecure, but there might be weaknesses. If not validating the server certificate in theory an active man in the middle could hijack the connection starting with the TLS handshake and thus change the parameters inside the TLS handshake. This includes protocol downgrades for using weak ciphers, weak protocol versions or switching off the requirement for a client certificate etc.

As long as client and server insist on strong ciphers and protocol version such downgrade will fail though. And as long the server enforces the use of a client certificate and has strict requirements on the client certificate (i.e. will not accept a fake one by the attacker) the active MITM attack will not work - see Do client certificates provide protection against MITM? for the details.

But again, it is best to use TLS for how it was designed. Because these use cases are focused on in terms of security and focused on in security research. Thus if server authentication is optional but client authentication required in your case then better switch the roles.

Steffen Ullrich
  • 190,458
  • 29
  • 381
  • 434
  • Just to add a small clarification: "... has strict requirements on the client certificate (i.e. will not accept a fake one by the attacker)." There can't really be "fake" credentials. In the current protocol, if a client is "trusted" by the server, the server stores their public key (hash). All connecting users provide a public key, but the server only treats them differently if it's a known key. (I believe keys are always used to be able to add connected users to the trusted list). My main goal is to prevent privilege escalation attacks. And the plan is to use TLS 1.3 only (or 1.2 minimum). – LRFLEW Dec 19 '20 at 09:38
  • BTW, I haven't marked this as the answer yet because it didn't address the question about the side effects of switching the client and server roles (eg. cipher suite negotiation). If you have any answers for that part, I'd appreciate it if you would be willing to add it to your response. If I don't get any other answers, I'll end up marking this as the answer, but I want to leave the question as "unanswered" for a little while longer to see if anybody has an answer for that part. – LRFLEW Dec 19 '20 at 22:57
  • @LRFLEW: The ultimate decision which cipher and protocol to use is **always** at the TLS server side, but is based on what the client offers. The option `SSL_OP_CIPHER_SERVER_PREFERENCE` only configures a specific server behavior, i.e. adhere to the cipher order preferred by the client or to its own order. It has nothing to do who has the TLS server or client role. – Steffen Ullrich Dec 19 '20 at 23:03
  • I think I might have caused confusion about what I'm asking for. I had asked "What other effects of switching the TLS roles should I consider?" Choosing the cipher suite is done by the TLS Server, so switching the roles affects who makes the choice. Are there any other parts of the TLS protocol *like this* that changing the parties' roles will affect? I want to keep any side effects in mind when making the decision on the protocol direction. – LRFLEW Dec 19 '20 at 23:08
  • @LRFLEW: While the TLS server has the ultimate decision about protocol and ciphers and can only do this within the limits of the TLS clients offer. Thus the TCP server (aka TLS client in the switched case) side can still enforce strong ciphers and protocol. And this use case is uncommon but not unknown: for example in FTPS active mode the FTP server creates a TCP data connection to the FTP client (i.e. FTP client is TCP server) and then the FTP client starts the TLS handshake over this (i.e. FTP client is TCP server but TLS client). – Steffen Ullrich Dec 20 '20 at 06:27
  • @LRFLEW: In general I suggest that you don't talk about switching roles at all but consider TLS and its direction independent from the underlying TCP connection and its direction. There are several protocols (SMTP, IMAP, FTP ...) which start with plain TLS but then after some time and command switch to TLS and might even switch back to plain TCP (as with FTP CCC command). Just treat TCP as the direction-agnostic transport layer and use client and server roles in TLS like you need it for your use case. – Steffen Ullrich Dec 20 '20 at 06:31