Access control for embedded device

Question

We develop a protocol for computer remote control of a certain kind of toy. It's basically a sensor and actor network that is usually connected to the user PC via serial port or USB. Now some users wish to connect via Ethernet, and we want to design some kind of gateway in an embedded device.

It is expected to be used only in home LANs (e.g. from mobile devices through a WiFi router), but those shouldn't be considered secure and there's always a chance of being exposed to the internet. The risks involved are manifold:

• those toys are expensive and could be destroyed through abuse
• the environment might be harmed by messing with the actor power supplies
• the devices are capable of (unsecured) firmware updates and could be taken over or bricked
• generic disturbance of service operation
• eavesdropping on private data

The embedded gateway device would expose some kind of server that accepts connections - which need to be limited to authorised ones. How to do this?

After the initial idea of wrapping our serial protocol in UDP packets (for low latency and low overhead), we considered TCP to avoid dealing with packet loss in wireless networks. TLS is the next logical step, which provides encryption¹. But how to do the authentication and authorisation?

Is TLS applicable at all, when used in a non-web context? We don't necessarily have host names, or even internet access, and I don't know how to validate the certificates. We cannot use a CA, so our PKI would look different and I don't know what security holes are introduced by manually managing certificates. It's probably already enough of an issue to get each server - each device - its own randomly generated certificate. (I also found these two questions dealing with similar issues)

Would some kind of pairing process (similar to Bluetooth or WPS) solve the issue with unvalidated certificates? The embedded device would be equipped with one simple push-button to accept a client², and the client would remember ("pin"?) the server certificate. This first connection would be susceptible to a MITM-attack, but subsequent connections would not. Does that sound viable and secure? I'm doubtful about home-brewed schemes.

An alternative to the pairing I considered would be a password authorisation scheme, where the client sends a password to the server to gain access.³ However, users tend to choose weak passwords, and also I have no idea how (i.e. which protocol to use) to send the password over the unsecured connection - still not having validated any certificates -, avoiding MITM or replay attacks.

Please help me find a good solution - either by answering the above questions or by suggesting a completely different approach that I missed entirely. Our main objectives/restrictions are:

• good usability even for non-technical users - without any administrative maintenance
• simple implementation with widely available libraries for many programming languages
• high level of security (if necessary ignoring setup phase where it might be impossible)

_{1: As long as used with the right settings, I gathered.
2: I guess we must not forget a way to remove clients
3: With the initial password randomly assigned to each device and printed on the bottom}

Answer 1

This is a rather broad question, so I cannot give more than hints. First at its lowest level, the SSL/TLS library (OpenSSL) only require that both parts involved in a communication have private and public keys and that each knows the public key of the other one. That is the way it is used for example in OpenSSH. The PKI infrastructures and CA and only a way to validate public keys for entities that do not know each other.

In your use case, you could use a (private) Certificate Authority to validate certificates for the devices and eventually certificates for the client PCs. But in case a key was compromised, the renewal of a certificate would be hard while it does not add much security to a mere public key exchange the way OpenSSL do.

So my advice would be:

• find/build an acceptable way to generate random key pairs on the devices and client software. The good point with it is that if the device is sold, the new owner can install a brand new key on it.
• imagine and develop a way to exchange public keys between a device and a PC. Something like broadcasting them in a time window followed by an agreement protocol if one and only one key has been received. After that point, the peer key should be pinned on each part

From now on, the device should just reject any connection request that cannot prove the knowledge of its peer key. That allows you to:

• rely on the high quality OpenSSL (or probably LibreSSL) library.
• allow the owner to change the keys on his device at will if it can suspect his PC or device could have been compromised, or if for any reason the device or the PC hast lost its previous key (for example is the device memory had to be replaced, or if the PC disk crashed)
• require no more that 2 buttons (generate a new key and start an association) on the device or even one single button if a new association can consistently use a new key

Alternatively, you could setup a special mode (activated by a button on device) in which any certificate will be accepted during a time window and that certificate will be pinned for further exchanges provided one and only one was received. In that mode only the connection should be possible and no command would be processed. This variant avoids to setup a key exchange protocol and simply rely on the ability of SSL to pass certificates. So you end on both side with:

• generation of a new self signed certificate
• on special mode allowing the pinning of a peer certificate
• all other exchanges rely on the certificate send being the same as the pinned certificate

I know that this is far from a definitive answer, but it could allow to ask more precise questions here or on StackOverflow...