Final report on Clang / LLD state

February 01, 2019 posted by Michał Górny

Starting this month, I will be focusing my effort on LLDB, the debugger of the LLVM toolchain, as work contracted by the NetBSD Foundation. In this entry, I would like to shortly summarize what I've been working on before and what I have been able to accomplish, and what I am going to do next.

[Read More] [0 comments]


Integration of the LLVM sanitizers with the base system

February 01, 2019 posted by Kamil Rytarowski

Over the past month I've merged the LLVM compiler-rt sanitizers (LLVM svn r350590) with the base system. I've also managed to get a functional set of Makefile rules to build all of them, namely:
  • ASan
  • UBSan
  • TSan
  • MSan
  • libFuzzer
  • SafeStack
  • XRay
In all supported variations and modes that are supported by the original LLVM compiler-rt package.[Read More] [0 comments]


The hardware-assisted virtualization challenge

January 30, 2019 posted by Kamil Rytarowski

Over two years ago, I made a pledge to use NetBSD as my sole OS and only operating system, and to resist booting into any other OS until I had implemented hardware-accelerated virtualization in the NetBSD kernel (the equivalent of Linux' KVM, or Hyper-V).

Today, I am here to report: Mission Accomplished!

It's been a long road, but we now have hardware-accelerated virtualization in the kernel! And while I had only initially planned to get Oracle VirtualBox working, I have with the help of the Intel HAXM engine (the same backend used for virtualization in Android Studio) and a qemu frontend, successfully managed to boot a range of mainstream operating systems.[Read More] [4 comments]


The first report on LLD porting

January 18, 2019 posted by Kamil Rytarowski

Prepared by Michał Górny (mgorny AT

LLD is the link editor (linker) component of Clang toolchain. Its main advantage over GNU ld is much lower memory footprint, and linking speed. It is of specific interest to me since currently 8 GiB of memory are insufficient to link LLVM statically (which is the upstream default).

The first goal of LLD porting is to ensure that LLD can produce working NetBSD executables, and be used to build LLVM itself. Then, it is desirable to look into trying to build additional NetBSD components, and eventually into replacing /usr/bin/ld entirely with lld.

In this report, I would like to shortly summarize the issues I have found so far trying to use LLD on NetBSD.

[Read More] [2 comments]


The process of upstreaming support to LLVM sanitizers has been finalized

January 03, 2019 posted by Kamil Rytarowski

I've finished the process of upstreaming patches to LLVM sanitizers (almost 2000LOC of local code) and submitted to upstream new improvements for the NetBSD support. Today out of the box (in unpatched version) we have support for a variety of compiler-rt LLVM features: ASan (finds unauthorized memory access), UBSan (finds unspecified code semantics), TSan (finds threading bugs), MSan (finds uninitialized memory use), SafeStack (double stack hardening), Profile (code coverage), XRay (dynamic code tracing); while other ones such as Scudo (hardened allocator) or DFSan (generic data flow sanitizer) are not far away from completeness.

The NetBSD support is no longer visibly lacking behind Linux in sanitizers, although there are still failing tests on NetBSD that are not observed on Linux. On the other hand there are features working on NetBSD that are not functional on Linux, like sanitizing programs during early initialization process of OS (this is caused by /proc dependency on Linux that is mounted by startup programs, while NetBSD relies on sysctl(3) interfaces that is always available).[Read More] [2 comments]


NetBSD entering 2019 with more complete LLVM support

December 30, 2018 posted by Kamil Rytarowski

Prepared by Michał Górny (mgorny AT

I'm recently helping the NetBSD developers to improve the support for this operating system in various LLVM components. As you can read in my previous report, I've been focusing on fixing build and test failures for the purpose of improving the buildbot coverage.

Previously, I've resolved test failures in LLVM, Clang, LLD, libunwind, openmp and partially libc++. During the remainder of the month, I've been working on the remaining libc++ test failures, improving the NetBSD clang driver and helping Kamil Rytarowski with compiler-rt.

[Read More] [0 comments]


The NetBSD support update before the LLVM-8.0 branching point

December 16, 2018 posted by Kamil Rytarowski

Prepared by Michał Górny (mgorny AT

I'm recently helping the NetBSD developers to improve the support for this operating system in various LLVM components. My first task in this endeavor was to fix build and test issues in as many LLVM projects as timely possible, and get them all covered by the NetBSD LLVM buildbot.

Including more projects in the continuous integration builds is important as it provides the means to timely catch regressions and new issues in NetBSD support. It is not only beneficial because it lets us find offending commits easily but also because it makes other LLVM developers aware of NetBSD porting issues, and increases the chances that the patch authors will fix their mistakes themselves.

[Read More] [0 comments]


Reducing the delta with upstream version of sanitizers

December 02, 2018 posted by Kamil Rytarowski

I've been actively working on reducing the delta with the local copy of sanitizers with upstream LLVM sources. Their diff has been reduced to less than 2000 Lines Of Code. I've pushed to review almost all of the local code and I'm working on addressing comments from upstream developers. [Read More] [0 comments]


NetBSD LLVM Sanitizers in The Bay Area

November 01, 2018 posted by Kamil Rytarowski

I have presented the state of NetBSD sanitizers during two conferences in the San Francisco Bay Area: Google Summer of Code Mentor Summit (Mountain View) and MeetBSDCa (Santa Clara, Intel Campus SC12). I've also made progress in upstreaming of our local patches to LLVM sanitizers and introducing generic NetBSD enhancements there.[Read More] [1 comment]


EuroBSDCon 2018 and NetBSD sanitizers

October 01, 2018 posted by Kamil Rytarowski

I presented the state of NetBSD sanitizers during EuroBSDCon 2018 held in Bucharest, Romania.

I gave two talks, one covered userland sanitizers and the other one kernel sanitizers. Unfortunately video recordings from the conference are not available, but I've uploaded my slides online:

[Read More] [0 comments]


EuroBSDCon 2018 travel report and obligatory pics

October 01, 2018 posted by Maya Rashish

This was my first big BSD conference. We also planned - planned might be a big word - thought about doing a devsummit on Friday. Since the people who were in charge of that had a change of plans, I was sure it'd go horribly wrong.

The day before the devsummit and still in the wrong country, I mentioned the hours and venue on the wiki, and booked a reservation for a restaurant.

It turns out that everything was totally fine, and since the devsummit was at the conference venue (that was having tutorials that day), they even had signs pointing at the room we were given. Thanks EuroBSDCon conference organizers!

At the devsummit, we spent some time hacking. A few people came with "travel laptops" without access to anything, like Riastradh, so I gave him access to my own laptop. This didn't hold very long and I kinda forgot about it, but for a few moments he had access to a NetBSD source tree and an 8 thread, 16GB RAM machine with which to build things.

We had a short introduction and I suggested we take some pictures, so here's the ones we got. A few people were concerned about privacy, so they're not pictured. We had small team to hold the camera :-)

At the actual conference days, I stayed at the speaker hotel with the other speakers. I've attempted to make conversation with some visibly FreeBSD/OpenBSD people, but didn't have plans to talk about anything, so there was a lot of just following people silently.
Perhaps for the next conference I'll prepare a list of questions to random BSD people and then very obviously grab a piece of paper and ask, "what was...", read a bit from it, and say, "your latest kernel panic?", I'm sure it'll be a great conversation starter.

At the conference itself, was pretty cool to have folks like Kirk McKusick give first person accounts of some past events (Kirk gave a talk about governance at FreeBSD), or the second keynote by Ron Broersma.

My own talk was hastily prepared, it was difficult to bring the topic together into a coherent talk. Nevertheless, I managed to talk about stuff for a while 40 minutes, though usually I skip over so many details that I have trouble putting together a sufficiently long talk.

I mentioned some of my coolest bugs to solve (I should probably make a separate article about some!). A few people asked for the slides after the talk, so I guess it wasn't totally incoherent.

It was really fun to meet some of my favourite NetBSD people. I got to show off my now fairly well working laptop (it took a lot of work by all of us!).

After the conference I came back with a conference cold, and it took a few days to recover from it. Hopefully I didn't infect too many people on the way back.



Using for Let's Encrypt certificates on servers

September 17, 2018 posted by S.P.Zeidler

Peter Wemm's writeup about using for served as inspiration, but I chose to do a few things different:
  • using DNS alias mode with sub-domains dedicated to ACME verification
  • delegating the sub-domains to the servers where the certificate will be needed
  • using bind on the servers where the certificate will be needed (where it was running as resolver already anyway)
  • using dns_nsupdate (i.e. dynamic DNS) to add the challenge to the ACME subzone.
Appropriately restricted, that gives the following addition to named.conf on the target server (with an update key named acme-ddns):
options {
        allow-update { localhost; };

zone "" {
        type master;
        file "acme/";
        update-policy {
                grant acme-ddns name TXT;
And last but not least, deployment of certificates via make, i.e. completely independent of

Due to all of the above, does not need to tentacle about in the filesystem and can run as a plain user in a chroot. It's not a tiny chroot, though (20M), since needs a bunch of common shell tools:

  • awk basename cat chmod cp curl cut date egrep/grep head mkdir mktemp mv nsupdate od openssl printf readlink rm sed sh sleep stat tail touch tr uname, and their shared libs, /libexec/ld.elf_so and /usr/libexec/ld.elf_so;
  • under the chroot /etc a resolv.conf, the CA cert for Let's Encrypt (mozilla-rootcert-60.pem) and to make openssl complain less an empty openssl.cnf
  • and in the chroot /dev: null, random and urandom.

I call both the --cron job and the certificate deployment make from daily.local, which adds the output to the daily mail and makes it easy to keep an eye on things. [0 comments]