GSoC Reports: Fuzzing Rumpkernel Syscalls, Part 2

I have been working on Fuzzing Rumpkernel Syscalls. This blogpost details the work I have done during my second coding period.

Reproducing crash found in ioctl()

Kamil has worked on reproducing the following crash

Thread 1 "" received signal SIGSEGV, Segmentation fault.
pipe_ioctl (fp=<optimized out>, cmd=<optimized out>, data=0x7f7fffccd700)
    at /usr/src/lib/librump/../../sys/rump/../kern/sys_pipe.c:1108
warning: Source file is more recent than executable.
1108                            *(int *)data = pipe->pipe_buffer.cnt;
(gdb) bt
#0  pipe_ioctl (fp=<optimized out>, cmd=<optimized out>, data=0x7f7fffccd700)
    at /usr/src/lib/librump/../../sys/rump/../kern/sys_pipe.c:1108
#1  0x000075b0de65083f in sys_ioctl (l=<optimized out>, uap=0x7f7fffccd820, retval=<optimized out>)
    at /usr/src/lib/librump/../../sys/rump/../kern/sys_generic.c:671
#2  0x000075b0de6b8957 in sy_call (rval=0x7f7fffccd810, uap=0x7f7fffccd820, l=0x75b0de126500, 
    sy=<optimized out>) at /usr/src/lib/librump/../../sys/rump/../sys/syscallvar.h:65
#3  sy_invoke (code=54, rval=0x7f7fffccd810, uap=0x7f7fffccd820, l=0x75b0de126500, sy=<optimized out>)
    at /usr/src/lib/librump/../../sys/rump/../sys/syscallvar.h:94
#4  rump_syscall (num=num@entry=54, data=data@entry=0x7f7fffccd820, dlen=dlen@entry=24, 
    retval=retval@entry=0x7f7fffccd810)
    at /usr/src/lib/librump/../../sys/rump/librump/rumpkern/rump.c:769
#5  0x000075b0de6ad2ca in rump___sysimpl_ioctl (fd=<optimized out>, com=<optimized out>, 
    data=<optimized out>) at /usr/src/lib/librump/../../sys/rump/librump/rumpkern/rump_syscalls.c:979
#6  0x0000000000400bf7 in main (argc=1, argv=0x7f7fffccd8c8) at test.c:15

in the rump using a fuzzer that uses pip2, dup2 and ioctl syscalls and specific arguments that are known to cause a crash upon which my work develops.

https://github.com/adityavardhanpadala/rumpsyscallfuzz/blob/master/honggfuzz/ioctl/ioctl_fuzz2.c

Since rump is a multithreaded process. Crash occurs in any of those threads. By using a core dump we can quickly investigate the crash and fetch the backtrace from gdb for verification however this is not viable in the long run as you would be loading your working directory with lots of core dumps which consume a lot of space. So we need a better way to reproduce crashes.

Crash Reproducers

Getting crash reproducers working took quite a while. If we look at HF_ITER() function in honggfuzz, it is a simple wrapper for HonggfuzzFetchData() to fetch buffer of fixed size from the fuzzer.

void HonggfuzzFetchData(const uint8_t** buf_ptr, size_t* len_ptr) {
.
.
.
.
    *buf_ptr = inputFile; 
    *len_ptr = (size_t)rcvLen;
.
.
}

And if we observe the attribute we notice that inputFile is mmaped.

//libhfuzz/fetch.c:26
    if ((inputFile = mmap(NULL, _HF_INPUT_MAX_SIZE, PROT_READ, MAP_SHARED, _HF_INPUT_FD, 0)) ==
        MAP_FAILED) {
        PLOG_F("mmap(fd=%d, size=%zu) of the input file failed", _HF_INPUT_FD,
            (size_t)_HF_INPUT_MAX_SIZE);
    }

So in a similar approach HF_ITER() can be modified to read input from a file and be mmapped so that we can reuse the reproducers generated by honggfuzz.

Attempts have been made to use getchar(3) for fetching the buffer via STDIN but for some unknown reason it failed so we switched to mmap(2)

So we overload HF_ITER() function whenever we require to reproduce a crash. I chose the following approach to use the reproducers. So whenever we need to reproduce a crash we just define CRASH_REPR.

static
void Initialize(void)
{
#ifdef CRASH_REPR
    FILE *fp = fopen(argv[1], "r+");
    data = malloc(max_size);
    fread(data, max_size, 1, fp);
    fclose(fp);
#endif
    // Initialise the rumpkernel only once.
    if(rump_init() != 0)
        __builtin_trap();
}

#ifdef CRASH_REPR
void HF_ITER(uint8_t **buf, size_t *len) {
        *buf = (uint8_t *)data;
        *len = max_size;
        return;
}
#else
EXTERN void HF_ITER(uint8_t **buf, size_t *len);
#endif

This way we can easily reproduce crashes that we get and get the backtraces.

Generating C reproducers

Now the main goal is to create a c file which can reproduce the same crash occuring due to the reproducer. This is done by writing all the syscall executions to a file with arguments so they can directly be compiled and used.

#ifdef CRASH_REPR
        FILE *fp = fopen("/tmp/repro.c","a+");
        fprintf(fp, "rump_sys_ioctl(%" PRIu8 ", %" PRIu64 ");\n",get_u8(),get_ioctl_request());
        fclose(fp);
#else    
        rump_sys_ioctl(get_u8(), get_ioctl_request());
#endif

I followed the same above method for all the syscalls that are executed. So I get a proper order of syscalls executed in native c code that I can simply reuse.

Obstacles

The number of times each syscall is executed before getting to a crash is quite high. So trying to perform a write to a file or STDOUT will create a lot of overhead when the number of syscalls executed are quite high. This method is good enough but a bit of optimization will make it even better.

To-Do

• ./build.sh building rump on linux+netbsd
• pregenerating fuzzer input using the implementation similar to that used in syzkaller.

Finally I thank my mentors Siddharth Muralee, Maciej Grochowski, Christos Zoulas for their guidance and Kamil Rytarowski for his constant support whenever I needed it.