NX bit: does it protect the stack?

Question

I once heard the NX bit was a panacea, then that it was not. One detail I've wondered about though: Does the NX (no execute) bit protect against code inserted into the stack and executed there? It seems to me that the stack, because it's not the heap, might not be protected by NX usually. THanks.

Answer 1

The NX bit is a feature of the Memory Management Unit of some CPU (including recent enough x86). It allows to mark each memory page as being "allowed" or "disallowed" for code execution. The MMU is under control of the kernel; the kernel code decides which pages get the execution privilege and which do not. Therefore, whether the stack space is protected against execution depends on the OS. It also depends on a lot of other things:

• Though the kernel may apply default values for the page properties, application can request changes. E.g. on Linux systems, the mprotect() system call can be used to enable or disable execution on any page that is under control of the calling process. In particular, any application which contains a JIT compiler (e.g. Web browsers, for efficient Javascript support) must play with mprotect(), because it must generate code (i.e. write bytes into some memory) and then execute the said code.

• A mono-threaded application gets its stack in a dedicated area that the kernel knows of; in particular, the stack pages are automatically allocated upon first usage. In a multi-threaded application, things change: each thread has its own stack, which is, from the point of view of the kernel, heap-allocated. Depending on the OS, there may or may not be special support from the kernel for thread stacks.

• GCC in particular supports a C language extension called nested functions. Because of the semantics of these functions, the compiled code for a nested function must dynamically produce some executable code on the stack; this is called a trampoline. See this page for some details. The bottom-line is that, for a trampoline to actually work, the stack (or at least the page in which the trampoline resides) must be marked executable.

It so happens that in the ELF file format, which is the format of executable files and DLL on Linux, there is a field which allows to specify whether the stack should be executable or not. When GCC compiles some code which contains a nested function, it sets this flag in the produced binary.

I have made some tests on a fairly recent Linux (Ubuntu 13.10, on 64-bit x86, with NX bit activated). Whether a given page is executable or not can be inferred from the contents of the special file /proc/$$/maps where $$ is the process ID. These tests have shown the following:

• By default, the main stack is not executable. The NX bit is set for the stack pages.
• If code contains a nested function, the executable is marked with the "executable stack" flag and, indeed, the main stack is now executable.
• Thread stacks are created with the same "executable status" as the main stack.
• If a "normal" executable, with its main stack non-executable, dynamically loads a DLL (with dlopen()) which contains code for a nested function, then the main stack is automatically switched to executable status.
• If a normal multi-threaded executable, with its main stack and all its thread stacks non-executable, dynamically loads a DLL which contains code for the nested function, then the main stack and all the thread stacks are en masse promoted to executable status. Note that this implies that on this Linux system, "something" (probably the kernel, I have not checked) is aware of all the current thread stacks, and can change their mapping rights dynamically.

From all of this we conclude that on at least some recent versions of Linux, the stacks (main and threads) will usually be marked as non-executable (i.e. NX bit set). However, if the executable code of the application, or some executable code somewhere in a DLL loaded by the application, contains a nested function or otherwise advertises a need for an executable stack, then all stacks in the application will be marked as executable (i.e. NX bit not set). In other words, a single loadable plugin or extension for an application may deactivate NX stack protection for all threads of the applications, simply by using a rarely used but legitimate and documented feature of GCC.

There is no need to panic, though, because, as @tylerl points out, protection afforded by the NX bit is not that great. It will make some exploits more awkward for the least competent of attackers; but good attackers will not be impeded. Also, all this talk about NX is about trying to contain damage once a buffer overflow or use-after-free has occurred, and that's arguably a bit late to react.

Answer 2

By marking the stack as non-execute, you effectively prevent code inserted into the stack from running. You're not protecting the stack from modification; rather, you're causing a hard crash when the code attempts to jump to a position in the NX-marked stack.

The workaround is to not attempt to execute code on the stack. Instead of setting the return position to a spot in the stack, you set it to the location of a system function such as exec(), setting the spots you overwrote on the stack to contain arguments to the exec() system call instead of executable machine code. In such a scenario, the attacker can run a system function with the parameters of his choosing, which is sufficient, for example, to execute another program.

NX therefore provides some level of protection, but not tons. It somewhat limits the flexibility of what an attacker can do with the initial exploit, but given he can leverage that exploit to execute a shell, there's not a lot of safety there.

The next layer in defense is ASLR, which randomizes the location of system functions such as exec(). This way, when he attempts to execute his shell, he won't know what address to load into the return location.

Attacks against ALSR are typically brute force; run your exploit multiple times, trying different return addresses each time. This means that ALSR is significantly less effective on 32-bit system than on 64-bit because of the limited range of randomization, allowing for relatively fast brute-force attacks.

ASLR combined with NX offers some degree of protection, but nothing is absolute.