Final report on Clang / LLD state


February 01, 2019 posted by Michał Górny

Starting this month, I will be focusing my effort on LLDB, the debugger of the LLVM toolchain, as work contracted by the NetBSD Foundation. In this entry, I would like to shortly summarize what I've been working on before and what I have been able to accomplish, and what I am going to do next.

[Read More] [0 comments]

 

The hardware-assisted virtualization challenge


January 30, 2019 posted by Kamil Rytarowski

Over two years ago, I made a pledge to use NetBSD as my sole OS and only operating system, and to resist booting into any other OS until I had implemented hardware-accelerated virtualization in the NetBSD kernel (the equivalent of Linux' KVM, or Hyper-V).

Today, I am here to report: Mission Accomplished!

It's been a long road, but we now have hardware-accelerated virtualization in the kernel! And while I had only initially planned to get Oracle VirtualBox working, I have with the help of the Intel HAXM engine (the same backend used for virtualization in Android Studio) and a qemu frontend, successfully managed to boot a range of mainstream operating systems.[Read More] [6 comments]

 

The first report on LLD porting


January 18, 2019 posted by Kamil Rytarowski

Prepared by Michał Górny (mgorny AT gentoo.org).

LLD is the link editor (linker) component of Clang toolchain. Its main advantage over GNU ld is much lower memory footprint, and linking speed. It is of specific interest to me since currently 8 GiB of memory are insufficient to link LLVM statically (which is the upstream default).

The first goal of LLD porting is to ensure that LLD can produce working NetBSD executables, and be used to build LLVM itself. Then, it is desirable to look into trying to build additional NetBSD components, and eventually into replacing /usr/bin/ld entirely with lld.

In this report, I would like to shortly summarize the issues I have found so far trying to use LLD on NetBSD.

[Read More] [2 comments]

 

NetBSD entering 2019 with more complete LLVM support


December 30, 2018 posted by Kamil Rytarowski

Prepared by Michał Górny (mgorny AT gentoo.org).

I'm recently helping the NetBSD developers to improve the support for this operating system in various LLVM components. As you can read in my previous report, I've been focusing on fixing build and test failures for the purpose of improving the buildbot coverage.

Previously, I've resolved test failures in LLVM, Clang, LLD, libunwind, openmp and partially libc++. During the remainder of the month, I've been working on the remaining libc++ test failures, improving the NetBSD clang driver and helping Kamil Rytarowski with compiler-rt.

[Read More] [0 comments]

 

The NetBSD support update before the LLVM-8.0 branching point


December 16, 2018 posted by Kamil Rytarowski

Prepared by Michał Górny (mgorny AT gentoo.org).

I'm recently helping the NetBSD developers to improve the support for this operating system in various LLVM components. My first task in this endeavor was to fix build and test issues in as many LLVM projects as timely possible, and get them all covered by the NetBSD LLVM buildbot.

Including more projects in the continuous integration builds is important as it provides the means to timely catch regressions and new issues in NetBSD support. It is not only beneficial because it lets us find offending commits easily but also because it makes other LLVM developers aware of NetBSD porting issues, and increases the chances that the patch authors will fix their mistakes themselves.

[Read More] [0 comments]

 

=?iso-8859-8-i?Q? Handling non-UTF-8 Hebrew email


June 10, 2018 posted by Maya Rashish

I like to use CLI email clients (mutt). This by itself is not unusual, but I happen to do this while speaking a language written right-to-left, Hebrew.
Decent bidi support in CLI tools is rare, so my impression is that very few people do this.[Read More] [1 comment]

 

Getting my new laptop to work


May 03, 2018 posted by Maya Rashish

I've recently been gifted a fancy laptop - a Dell XPS 15 9550.
I want to run NetBSD on it and have it run well, and I've set aside time to achieve this.
These are some of the lessons I learned from porting code to support my SD card reader and wireless card.

[Read More] [0 comments]

 

Mercurial mirror on Bitbucket


September 01, 2017 posted by Kamil Rytarowski

Joerg Sonnenberger has announced a new set of mirrored repositories.

You can find Mercurial versions of src, pkgsrc and xsrc under

and

The same rules as for the fossil and github repositories apply, i.e. there may be occasional glitches and if it becomes too bad, they might be recreated from scratch.

See more information in the posted thread to tech-repository. [0 comments]

 

New home for the repository conversion


June 10, 2017 posted by Kamil Rytarowski

Hello all,

the repository conversion setup for NetBSD CVS -> Fossil -> Git has found a new home. Ironically, on former cvs.NetBSD.org hardware. This provides a somewhat faster conversion cycle as well as removing anoncvs.NetBSD.org from the process. This should avoid occasional problems with incomplete syncs. Two other changes have been applied at the same time:[Read More] [0 comments]

 

NetBSD maintainer in the QEMU project


May 17, 2017 posted by Kamil Rytarowski

QEMU - the FAST! processor emulator - is a generic, Open Source, machine emulator and virtualizer. It defines state of the art in modern virtualization.

This software has been developed for multiplatform environments with support for NetBSD since virtually forever. It's the primary tool used by the NetBSD developers and release engineering team. It is run with continuous integration tests for daily commits and execute regression tests through the Automatic Test Framework (ATF).[Read More] [2 comments]

 

New synchronization mechanism - localcount(9)


May 03, 2017 posted by Paul Goyette

A new localcount(9) reference-counting mechanism will soon be available to provide improved protection against having a device or driver "disappear" while it is being used. [Read More] [0 comments]

 

First reproducible builds conference in Athens


December 14, 2015 posted by Thomas Klausner

Last week I met with about 40 other developers from various projects (mostly Debian, but also Arch Linux, FreeBSD, Guix, Homebrew, MacPorts, Tor and some others) in Athens for a three day conference about reproducible builds, i.e. the task of getting the same binaries from the same source on a particular platform.

The advantages are better verifyability that the source code matches the binaries, thus addressing one of the many steps one has to check before trusting the software one runs.

We discussed various topics during the conference in small groups:

  • technical aspects (how to achieve this, how to cooperate over distributions, ...)
  • social aspects (how to argue for it with programmers, managers, lay people) financial aspects (how to get funding for such work)
  • lots of other stuff :)
For NetBSD, there are two parts:

Making the base system reproducible: a big part of the work for this has already been done, but there a number of open issues, visible e.g. in Debian's regularly scheduled test builds, up to the fact that this is not the default yet.

Making pkgsrc reproducible: This will be a huge task, since pkgsrc targets so many and diverse platforms. On the other hand, we have a very good framework below that that should help.

For giggles, I've compared the binary packages for png built on 7.99.22 and 7.99.23 (in my chrooted pbulk only though) and found that most differences were indeed only timestamps. So there's probably a lot of low-hanging fruit in this area as well.

If you want to help, here are some ideas:

  • fix the MKREPRO bugs (like PRs 48355, 48637, 48638, 50119, 50120, 50122)
  • check https://reproducible.debian.net/netbsd/netbsd.html for more issues, or do your own tests
  • discuss turning on MKREPRO by default
  • starting working on reproducibility in pkgsrc:
    • remove gzip time stamps from binary packages
    • use a fixed time stamp for files inside binary packages (perhaps depending on newest file in sources, or latest change in pkgsrc files for the pkg)
    • identify more of the issues, like how to get symbols ordered reproducible in binaries (look at shells/bash)
Thanks to the NetBSD developers who already worked on this before, and to TNF for funding the travel and the Linux Foundation for funding the accomodation for my participation in the conference, and Holger Levsen for inviting me. [0 comments]