It is possible to create a secure system this way using RSA, but you're looking for a world of pain and it's likely you'll end up creating an insecure system.
RSA is a very low level cryptographic primitive, it's way too low level for an application developer to work with and using it exposes you to a lot of vulnerabilities if not used correctly. Leave writing cryptographic code to the experts and use higher level cryptography, either TLS or OpenPGP should be the level that you're looking at, not RSA.
I know that there are a performance issue with RSA, but it is for very short messages, so it is not concerning me.
The issue with RSA isn't just performance. RSA cannot be used to encrypt data larger than the key size minus any paddings, which means that a 2048-bit RSA key has a maximum input size of 256 bytes (or 245 bytes if you use v1.5 padding). This means that if you want to encrypt data larger than this size, you need to implement a chaining mode, and chaining mode is one of the most easy things to get wrong when implementing cryptography. Additionally nobody had ever implemented or does any cryptoanalysis on RSA chaining mode (most chaining mode are analysed for symmetric ciphers), so you are on your own there.
Another problem is that all cryptographic primitives eventually need to be replaced as computers get stronger and vulnerabilities are find in these primitives. Therefore, all practical cryptography applications need to add metadata that allow negotiation and replacement of the encryption primitives and its parameters. TLS, S/MIME, OpenPGP, and the crypt password hash format all have this capability built into the protocol. This is also another aspect that's easy to get wrong.
Another issue that plain RSA doesn't do is replay prevention. For this, you need to add a salt or IV, and possibly a challenge. And the order you layer signing and encryption also comes with their own pitfalls.
I could go all days explaining the various pitfalls if you're trying to use plain RSA to implement any cryptography system.
High level protocols like OpenPGP and TLS uses RSA as the underlying primitive but when you use a high level cryptographic protocols, those issues have been thought out for you by expert cryptographers and their implementations have been highly scrutinized as well. All the remaining issues are either resolved or it's fairly easy to configure the system when there's a trade off.
OpenPGP in particular have a signing mode, which effectively does what you mentioned, using private key to sign a data to prove authorship, though in more sophisticated ways. An OpenPGP signature is created by encrypting a hash of the message rather than the message itself, thus avoiding the need to do chaining and it adds a number of metadata to protect against various attacks. TLS have similar properties during handshake if you use Mutual Authentication but they are implicit in the connection, so it may not be suitable if you want to store and/or forward the proof of authorship to another entity.
TL;DR Don't roll your own cryptography. If you need to persist the proof of authorship, use OpenPGP signature; otherwise just wrap the communication in TLS using Mutual Authentication.
Why others not doing it? There is any security issue with it?
Because it's stupidly complicated to get it right, when you can just use TLS mutual authentication with just a few lines of code. Unless your scheme is going to be as massively scrutinized as TLS and OpenPGP, I can assure you your scheme will have weaknesses you would have practically no chance to discover.