NetBSD maintainer in the QEMU project

May 17, 2017 posted by Kamil Rytarowski

QEMU - the FAST! processor emulator - is a generic, Open Source, machine emulator and virtualizer. It defines state of the art in modern virtualization.

This software has been developed for multiplatform environments with support for NetBSD since virtually forever. It's the primary tool used by the NetBSD developers and release engineering team. It is run with continuous integration tests for daily commits and execute regression tests through the Automatic Test Framework (ATF).[Read More] [1 comment]


New synchronization mechanism - localcount(9)

May 03, 2017 posted by Paul Goyette

A new localcount(9) reference-counting mechanism will soon be available to provide improved protection against having a device or driver "disappear" while it is being used. [Read More] [0 comments]


First reproducible builds conference in Athens

December 14, 2015 posted by Thomas Klausner

Last week I met with about 40 other developers from various projects (mostly Debian, but also Arch Linux, FreeBSD, Guix, Homebrew, MacPorts, Tor and some others) in Athens for a three day conference about reproducible builds, i.e. the task of getting the same binaries from the same source on a particular platform.

The advantages are better verifyability that the source code matches the binaries, thus addressing one of the many steps one has to check before trusting the software one runs.

We discussed various topics during the conference in small groups:

  • technical aspects (how to achieve this, how to cooperate over distributions, ...)
  • social aspects (how to argue for it with programmers, managers, lay people) financial aspects (how to get funding for such work)
  • lots of other stuff :)
For NetBSD, there are two parts:

Making the base system reproducible: a big part of the work for this has already been done, but there a number of open issues, visible e.g. in Debian's regularly scheduled test builds, up to the fact that this is not the default yet.

Making pkgsrc reproducible: This will be a huge task, since pkgsrc targets so many and diverse platforms. On the other hand, we have a very good framework below that that should help.

For giggles, I've compared the binary packages for png built on 7.99.22 and 7.99.23 (in my chrooted pbulk only though) and found that most differences were indeed only timestamps. So there's probably a lot of low-hanging fruit in this area as well.

If you want to help, here are some ideas:

  • fix the MKREPRO bugs (like PRs 48355, 48637, 48638, 50119, 50120, 50122)
  • check for more issues, or do your own tests
  • discuss turning on MKREPRO by default
  • starting working on reproducibility in pkgsrc:
    • remove gzip time stamps from binary packages
    • use a fixed time stamp for files inside binary packages (perhaps depending on newest file in sources, or latest change in pkgsrc files for the pkg)
    • identify more of the issues, like how to get symbols ordered reproducible in binaries (look at shells/bash)
Thanks to the NetBSD developers who already worked on this before, and to TNF for funding the travel and the Linux Foundation for funding the accomodation for my participation in the conference, and Holger Levsen for inviting me. [0 comments]


Unbloating the VAX install CD

June 08, 2014 posted by Martin Husemann

We all know that NetBSD-current is bloated compared to VMS of the early 80s. But we still can boot on some VAXes, so why not make it as easy as possible to install?[Read More] [2 comments]


First ports switched to gcc 4.8

March 06, 2014 posted by Martin Husemann

NetBSD starts deploying gcc 4.8[Read More] [1 comment]


Google Summer of Code 2013 report: Defragmentation for FFS

October 11, 2013 posted by Thomas Klausner

The following report is by Manuel Wiesinger:

First of all, I like to thank the NetBSD Foundation for enabling me to successfully complete this Google Summer of Code. It has been a very valuable experience for me.

My project is a defragmentation tool for FFS. I want to point out at the beginning that it is not ready for use yet.

What has been done:

Fragment analysis + reordering. When a file is smaller or equal than the file system's fragment size, it is stored as a fragment. One can think of a fragment as a block. It can happen that there are many small files that occupy a fragment. When the file systems changes over time it can happen that there are many blocks containing fewer fragments than they can hold. The optimization my tool does is to pack all these fragments into fewer blocks. This way the system may get a little more free space.

Directory optimization. When a directory gets deleted, the space for that directory and its name are appended to the previous directory. This can be imagined like a linked list. My tool reads that list and writes all entries sequentially.

Non-contiguous files analysis + reordering strategy. This is what most other operating systems call defragmentation - a reordering of blocks, so that blocks belonging to the same file or directory can be read sequentially.

[Read More] [1 comment]


Firefox on sparc64 update

September 23, 2013 posted by Martin Husemann

Just a small update on the previous post about firefox on sparc64: after a bit more work, the brand new version 24 ESR builds straight from pkgsrc (so should be included in the next set of binary pkgs).

All open issues (wrong colours on scaled images, failing https, ...) have been resolved.

Here is a new screeenshot:



Firefox on sparc64

May 26, 2013 posted by Martin Husemann

New firefox will be available for NetBSD/sparc64 again starting with the import of the official version 22 release into pkgsrc.[Read More] [4 comments]


Initial Kyua import done

March 01, 2013 posted by Julio Merino

I am very happy to announce that the initial import of Kyua into the NetBSD source tree is complete! I am sure there are some rough edges and here is where you come into play.

First of all, let me clarify that all the integration changes are gated by the MKKYUA build setting, which still defaults to no. Unless you explicitly set MKKYUA=yes in /etc/mk.conf or in the command line, you will not get a system with Kyua. However, once you set that flag, you will transition to the full new setup:

  • Kyua will be installed as /usr/bin/kyua. See kyua(1) to get started.
  • The old atf-run and atf-report tools will become compatibility wrappers. These should be a reasonable drop-in replacement for most use cases, but they are probably not perfect.
  • Your /usr/tests tree will be populated by a bunch of Kyuafiles.

So what's new compared to the ATF tools? Here are some highlights:

  • Ability to write and run ATF-less tests. It has been a common desire around here to develop test programs that do not rely on the ATF libraries. With this change, this becomes possible. See kyuafile(5) for details.
  • Direct HTML report generation. There is no need to set up a complex XML toolchain any more to convert the ATF test reports into HTML pages. Kyua can generate plain HTML directly, so it will be feasible to serve such content straight from NetBSD's built-in httpd.
  • Historical data. As seen in the various test beds that have appeared around ATF, there is a desire to maintain historical data of the test results. Kyua does that natively, by recording the results of the execution in a SQLite database. Reports can later be extracted from this database. There is still a lot of room for improvement here.
  • More flexible metadata and configuration. While this does not provide a real advantage today, as soon as the old atf-run and atf-report tools are gone we can trivially fix some long-standing issues (e.g. the inability to customize test deadlines).
  • Less complexity during test case execution. As you may have noticed over the years, the code in atf-run to capture the output of tests and deal with interrupts is not particularly robust. There have been several problems in this area, and I'm not convinced that they are all fixed. The new code works in a different manner and has been more carefully thought around these edges.
  • Independent testers. The code that implements the isolation of test cases and their controlled execution has been split into a set of "testers" that live in /usr/libexec/kyua-*-tester. These tools provide scriptable interfaces to interact with tests, with the idea that the kyua(1) frontend should end up being pretty straightforward. Should you want to write your own trivial script to run tests without kyua(1), you could pretty easily do that by interacting with the testers directly.

How can you help? Easy. Just rebuild your system with MKKYUA=yes, read through kyua(1), start using it to stress-test your system and report any problems you may encounter!

My immediate next steps include addressing your feedback and working with our major test runners to add support to their systems to use the new tools (for example, change anita to support running tests with kyua(1)).

Enjoy and thanks for reading. [0 comments]


Wanna Raspberry?

July 30, 2012 posted by Mike M. Volokhov

Demand for a cheap yet powerful computer is pretty high in geek circles. Remember those days when you could take a ZX Spectrum, or Commodore 64, or something like that and attach it to your TV to fill next few days with a great fun. Since then, PCs pushed out these tiny home computers. But today, Raspberry Pi — a little credit-card sized sweet computer — is here. Nick's Raspberry Pi

The Raspberry Pi is built on Broadcom BCM2835 system on chip which contains ARM1176 core running at 700MHz, with VideoCore 4 GPU, and has 256 MB of RAM on board. It provides two USB ports, 100Mb/s Ethernet, HDMI and composite video; audio output presented as well. Operating system loaded from SD flash.

Yet more fun is that the initial NetBSD support was just committed to the NetBSD source tree, and the Raspberry Pi can boot into multiuser now. Currently work on device drivers is in progress. USB and Ethernet are planned to be supported next.

Generally, NetBSD support for the ARM1176 core is in very good shape. The largest parts of the code I committed were the update to the plcom serial driver and the infrastructure to support the low level parts of the bcm2835 (interrupt controller, bus_space, and timer) — says Nick Hudson, who did the bringup. However, the porting has some hard nuts to crack: The graphics part is a bit of a challenge. But I hope to get dumb framebuffer support relatively soon. There is a publicly available datasheet for part of bcm2835, but certainly not the video controller.

Matt Thomas, the port-evbarm portmaster, was very knowledgeable and helpful in answering all my questions. I'd like to also thank Stephen Borrill for getting me an RPI in the first place. Stephen spoke to Eben Upton in Cambridge and soon after an RPI arrived — adds Nick.

So, there is a boot log from the very first NetBSD driven RPI board, and work on the device is in full swing. Many people exposed interest, and eventually the Raspberry Pi could became a good alternative for number of well supported, but aging single board computers.



Prettifying the NetBSD console

April 01, 2012 posted by Michael Lorenz

Over the last few months, many of NetBSD's graphical console drivers have gained the ability to use anti-aliased fonts. This was done mostly as a way to allow relatively easy access to a vast number of fonts, but they also look a lot better than plain monochrome fonts. With the genfb(4) driver this feature is available on almost all supported hardware which supports graphics. Some specialized drivers also support it (such as radeonfb(4), r128fb(4), voodoofb(4), ffb(4) and voyagerfb(4) ) and more are being added over time.

In order to improve overall usability and to give NetBSD a more unified look across different platforms, we are going to use a single default font and colour scheme on all platforms. Since studies prove its superiour readability we are proud to announce that Microsoft has granted permission to use the Comic Sans font in all future NetBSD releases.




posix_spawn syscall added

February 26, 2012 posted by Martin Husemann

The posix_spawn kernel implementation, mostly developed by Charles Zhang during Google Summer of Code 2011, has now been committed.

[Read More] [0 comments]