The use of public-key-encrypted algorithms within licensed applications can prevent hackers from cracking the licensed algorithms. Moreover, you can use them to disable licensed features that a user doesn't purchase. Licensing schemes eventually arrive at some critical decision point at which an algorithm decides whether the user is entitled to run that application. If not, the algorithm usually generates a console message or a message box to notify the user of his or her ineligibility. Then, the application terminates or continues in a feature-limited mode. The decision code generally resolves to one of the following (or similar) commands in assembly code:

Jump/Branch if not equal (for example, JNE, BNE)

Jump/Branch if equal (for example, JE, BEQ).

Typically, a hacker searches a disassembled listing for the termination message, finds the routine that displays the message, and then traces backward through the calling sequence until he locates the decision code. Often, changing a single hex byte in a binary image of the executable code completely bypasses the decision logic. Or changing the byte can reverse the decision logic. (The new copy runs only if it does not have a license.) The edited file can then generate a new .exe file. For example, you could replace the hex code for a JNE with a JMP (jump always) or a JE. To thwart hackers,

1. Select the code, C_PT, consisting of a single routine or a block of contiguous routines, that implements the licensing scheme.
2. Compile the application, linking in temporary code C_EN that generates a CRC of the associated object bytes and then compress and encrypt the associated object bytes. Then, dump the output C_CT to a file formatted for direct pasting into a C/C++ file. This file serves to initialize a const char array called EncryptedRoutines.
3. Now, insert runtime code, C_DE, that copies the contents of EncryptedRoutines to a buffer and then decrypts and decompresses the contents. Also, remove the original unencrypted code, C_PT, and the temporary code, C_EN.
4. Finally, recompile the application to link in C_DE and C_CT and to exclude C_EN and C_PT.

When the application starts up, it executes code C_DE, which copies the encrypted bytes from the EncryptedRoutines array into a dynamically created buffer. It then decrypts the buffer's contents using a public key, K₂, which the vendor provides, followed by decompression. The routine computes a CRC of the decrypted block. If this CRC matches the original CRC, the application then constructs a pointer to a decrypted entry-point function and uses the pointer to execute that function. CRCs that differ from each other indicate the use of the wrong public key, and the application terminates, because executing the still-encrypted code could be catastrophic. As a result, any system calls made from within the encrypted code do not show up in any disassembled listing. A hacker can detect them only if he or she is willing to acquire and use an emulator and trace through the execution.

Compression of the code to be encrypted—before encryption—prevents the hacker from simply replacing the encrypted bytes with the decrypted bytes. Now, even if the hacker recovers the decrypted bytes, he cannot subvert or change the licensing logic and then encrypt and replace the existing encrypted bytes, because he does know the private key, K₁. You should also encrypt all messages that the encrypted routines use. Most compilers for Windows/DOS generate position-independent code that results in relative calls: Jumps and Branches. When the system compiles and links the original application, the compiler assumes that all code executes from code space. Subroutine calls compile into PC-relative call instructions with the correct displacements. However, the decrypted routines run out of dynamically allocated data memory, so the displacements are all wrong.

The solution is to force all subroutine calls from the decrypted routines to be made to absolute addresses. However, the absolute addresses of all linked functions depend on the compiler and the options used, the order of the linked libraries, and the size of the user code, which will change because of Steps 2 to 4. Three solutions to these problems are:

1. If necessary, pad the code with dummy instructions, such that the size of C_PT+C_EN equals the size of C_DE+C_CT.
2. Link all subroutines called from within the decrypted code to fixed, user-specified addresses that will not change because of Steps 2 to 4.
3. Use an encrypted table of absolute-runtime-function addresses.

You can use the same concepts to protect code that implements a licensed feature. If you enter the correct public key, the decrypted code is usable. If you enter the wrong public key, the "decrypted" code remains encrypted, rendering that function unusable. You should use a unique private/public key pair for each licensed feature. Listing 1 shows Borland 5.02 code fragments that illustrate the concept of running decrypted code from dynamically allocated memory. The programmer can decide the implementation details of the CRC generation, compression, and private/public key algorithms.

Is this the best Design Idea in this issue? Select at www.ednmag.com.