LLDB work concluded

Upstream describes LLDB as a next generation, high-performance debugger. It is built on top of LLVM/Clang toolchain, and features great integration with it. At the moment, it primarily supports debugging C, C++ and ObjC code, and there is interest in extending it to more languages.

In February 2019, I have started working on LLDB, as contracted by the NetBSD Foundation. So far I've been working on reenabling continuous integration, squashing bugs, improving NetBSD core file support, extending NetBSD's ptrace interface to cover more register types and fix compat32 issues, fixing watchpoint and threading support, porting to i386.

March 2020 was the last month of my contract. During it my primary focus was to prepare integration of LLDB into NetBSD's src tree.

LLDB integration

The last important goal for the contract was to include LLDB in the NetBSD src tree. This mainly involved porting LLDB build into NetBSD src tree Makefiles. The resulting patches were sent to the tech-toolchain mailing list: [PATCH 0/7] LLDB import to src.

My proposed integration is based on LLDB tree from 2019-10-29. This matches the LLVM/Clang version currently imported in NetBSD. Newer version can not be used directly due to API incompatibility between the projects, and it is easier to backport LLDB fixes than to fix LLVM API missync.

The backports applied on top of this commit include all my contracted work, plus Kamil Rytarowski's work on LLDB. This also includes necessary fixes to make LLDB build against current NetBSD ptrace() API. Two source files in liblldbHost are renamed to ensure unique filenames within that library, as necessary to build from NetBSD Makefiles without resorting to ugly hacks.

Upstream uses to build individual LLDB components and plugins into split static libraries, then combine them all into a shared liblldb.so library. Both lldb and lldb-server executables link to it. We currently can not follow this model as LLVM and Clang sources are built without -fPIC and therefore are not suitable for shared libraries.

Therefore, we build everything as static libraries instead. This causes the logic that upstream uses to find lldb-server to fail, as it relies on obtaining the library path from the dynamic loader and finding executables relative to it. I have replaced it with hardcoded path to make LLDB work.

The patches are currently waiting for Joerg Sonnenberger to finish LLVM/Clang update that's in progress already.

Pending tasks

The exact list of pending tasks from my contract follows:

1. Add support to backtrace through signal trampoline and extend the support to libexecinfo, unwind implementations (LLVM, nongnu). Examine adding CFI support to interfaces that need it to provide more stable backtraces (both kernel and userland).

2. Add support for aarch64 target.

3. Stabilize LLDB and address breaking tests from the test suite.

Notes on backtracing through signal trampoline

I have described the problem of backtracing through signal trampoline in February's report. I haven't managed to finish the work on the topic within the contract but I will try to work on it in my free time.

Most likely, the solution would involve modifying the assembly in lib/libc/arch/*/sys/__sigtramp2.S. As suggested by Andrew Cagney, the CFI directives for amd64 would look like:

NENTRY(__sigtramp_siginfo_2)
    .cfi_startproc
    .cfi_signal_frame
    .cfi_def_cfa r15, 0
    /* offsets from mcontext_t */
    .cfi_offset rax, 0x70
    .cfi_offset rbx, 0x68
    .cfi_offset rcx, 0x18
    .cfi_offset rdx, 0x10
    /* ... */
    .cfi_def_cfa rsp, 8
    movq    %r15,%rdi
    movq    $SYS_setcontext, %rax
    syscall
    movq    $-1,%rdi /* if we return here, something is wrong */
    movq    $SYS_exit, %rax
    syscall
    .cfi_endproc
END(__sigtramp_siginfo_2)

Addressing breaking tests

While the most important functions of LLDB work on NetBSD, there are still many test failures. At this moment, there are 80 instances of @expectedFailureNetBSD decorator and 18 cases of @skipIfNetBSD. The former generally indicates that the test reliably fails on NetBSD and needs a fix, the latter is sometimes used to decorate tests specific to other systems but also to indicate that the test crashes, hangs or otherwise can not be reliably run.

Some tests are failing due to the concurrent signal kernel bug explained in the previous post and covered by XFAIL-ing ATF tests.

New regressions both in LLDB and in LLVM in general appear every month. Most of them are fixed by their authors once we report them. I will continue fighting new bugs in my free time and trying to keep the build bot green.

This work is sponsored by The NetBSD Foundation

The NetBSD Foundation is a non-profit organization and welcomes any donations to help us continue funding projects and services to the open-source community. Please consider visiting the following URL to chip in what you can:

https://netbsd.org/donations/#how-to-donate