Secure Programming Cookbook for C and C++

7.11. Performing Raw Decryption Using an RSA Private Key

Problem

You have a session key encrypted with an RSA public key (probably using a standard padding algorithm), and you need to decrypt the value with the corresponding RSA private key.

Solution

Your cryptographic library should have a straightforward API-to-RSA decryption algorithm: you should be able to give it the public key, the data to decrypt, a buffer for the results, and a specification as to what kind of padding was used for encryption (EME-OAEP padding is recommended; see Recipe 7.10). The size of the input message will always be equal to the bit length of RSA you’re using. The API function should return the length of the result, and this length will usually be significantly smaller than the input.

If, for some reason, you need to implement RSA on your own (which we strongly recommend against), refer to the Public Key Cryptography Standard (PKCS) #1, Version 2.1 (the latest version).

Discussion

Warning

While RSA is believed to be secure if used properly, it is very easy to use improperly. Be sure to read the Recipe on RSA encryption and the general-purpose considerations for public key encryption in Recipe 7.1 and Recipe 7.2 in addition to this one.

When using OpenSSL, decryption can be done with the RSA_private_decrypt( ) function, defined in openssl/rsa.h and shown below. It will return the length of the decrypted string, or -1 if an error occurs.

int RSA_private_decrypt(int l, unsigned char *ct, unsigned char *pt, RSA ...