Initial Kyua import done

I am very happy to announce that the initial import of Kyua into the NetBSD source tree is complete! I am sure there are some rough edges and here is where you come into play.

First of all, let me clarify that all the integration changes are gated by the MKKYUA build setting, which still defaults to no. Unless you explicitly set MKKYUA=yes in /etc/mk.conf or in the build.sh command line, you will not get a system with Kyua. However, once you set that flag, you will transition to the full new setup:

• Kyua will be installed as /usr/bin/kyua. See kyua(1) to get started.
• The old atf-run and atf-report tools will become compatibility wrappers. These should be a reasonable drop-in replacement for most use cases, but they are probably not perfect.
• Your /usr/tests tree will be populated by a bunch of Kyuafiles.

So what's new compared to the ATF tools? Here are some highlights:

• Ability to write and run ATF-less tests. It has been a common desire around here to develop test programs that do not rely on the ATF libraries. With this change, this becomes possible. See kyuafile(5) for details.
• Direct HTML report generation. There is no need to set up a complex XML toolchain any more to convert the ATF test reports into HTML pages. Kyua can generate plain HTML directly, so it will be feasible to serve such content straight from NetBSD's built-in httpd.
• Historical data. As seen in the various test beds that have appeared around ATF, there is a desire to maintain historical data of the test results. Kyua does that natively, by recording the results of the execution in a SQLite database. Reports can later be extracted from this database. There is still a lot of room for improvement here.
• More flexible metadata and configuration. While this does not provide a real advantage today, as soon as the old atf-run and atf-report tools are gone we can trivially fix some long-standing issues (e.g. the inability to customize test deadlines).
• Less complexity during test case execution. As you may have noticed over the years, the code in atf-run to capture the output of tests and deal with interrupts is not particularly robust. There have been several problems in this area, and I'm not convinced that they are all fixed. The new code works in a different manner and has been more carefully thought around these edges.
• Independent testers. The code that implements the isolation of test cases and their controlled execution has been split into a set of "testers" that live in /usr/libexec/kyua-*-tester. These tools provide scriptable interfaces to interact with tests, with the idea that the kyua(1) frontend should end up being pretty straightforward. Should you want to write your own trivial script to run tests without kyua(1), you could pretty easily do that by interacting with the testers directly.

How can you help? Easy. Just rebuild your system with MKKYUA=yes, read through kyua(1), start using it to stress-test your system and report any problems you may encounter!

My immediate next steps include addressing your feedback and working with our major test runners to add support to their systems to use the new tools (for example, change anita to support running tests with kyua(1)).

Enjoy and thanks for reading. [0 comments]

 

Wanna Raspberry?

Demand for a cheap yet powerful computer is pretty high in geek circles. Remember those days when you could take a ZX Spectrum, or Commodore 64, or something like that and attach it to your TV to fill next few days with a great fun. Since then, PCs pushed out these tiny home computers. But today, Raspberry Pi — a little credit-card sized sweet computer — is here.

The Raspberry Pi is built on Broadcom BCM2835 system on chip which contains ARM1176 core running at 700MHz, with VideoCore 4 GPU, and has 256 MB of RAM on board. It provides two USB ports, 100Mb/s Ethernet, HDMI and composite video; audio output presented as well. Operating system loaded from SD flash.

Yet more fun is that the initial NetBSD support was just committed to the NetBSD source tree, and the Raspberry Pi can boot into multiuser now. Currently work on device drivers is in progress. USB and Ethernet are planned to be supported next.

Generally, NetBSD support for the ARM1176 core is in very good shape. The largest parts of the code I committed were the update to the plcom serial driver and the infrastructure to support the low level parts of the bcm2835 (interrupt controller, bus_space, and timer) — says Nick Hudson, who did the bringup. However, the porting has some hard nuts to crack: The graphics part is a bit of a challenge. But I hope to get dumb framebuffer support relatively soon. There is a publicly available datasheet for part of bcm2835, but certainly not the video controller.

Matt Thomas, the port-evbarm portmaster, was very knowledgeable and helpful in answering all my questions. I'd like to also thank Stephen Borrill for getting me an RPI in the first place. Stephen spoke to Eben Upton in Cambridge and soon after an RPI arrived — adds Nick.

So, there is a boot log from the very first NetBSD driven RPI board, and work on the device is in full swing. Many people exposed interest, and eventually the Raspberry Pi could became a good alternative for number of well supported, but aging single board computers.

[23 comments]

 

Prettifying the NetBSD console

Over the last few months, many of NetBSD's graphical console drivers have gained the ability to use anti-aliased fonts. This was done mostly as a way to allow relatively easy access to a vast number of fonts, but they also look a lot better than plain monochrome fonts. With the genfb(4) driver this feature is available on almost all supported hardware which supports graphics. Some specialized drivers also support it (such as radeonfb(4), r128fb(4), voodoofb(4), ffb(4) and voyagerfb(4) ) and more are being added over time.

In order to improve overall usability and to give NetBSD a more unified look across different platforms, we are going to use a single default font and colour scheme on all platforms. Since studies prove its superiour readability we are proud to announce that Microsoft has granted permission to use the Comic Sans font in all future NetBSD releases.

Screenshots:

• a newly booted system using the new console
• showing output from the top utility.

[9 comments]

 

posix_spawn syscall added

The posix_spawn kernel implementation, mostly developed by Charles Zhang during Google Summer of Code 2011, has now been committed.

[Read More] [0 comments]

 

Creating atf-based tests for NetBSD src (new tutorial)

Dear readers,

With this post, I would like to announce the availability of the atf tutorial published on the wiki!

This new document intends to aid NetBSD developers in writing atf-based tests for the NetBSD source tree. It does so by providing a description of the basic concepts behind the atf design and listing what the necessary steps are to write an atf-based test for the NetBSD source tree. The tutorial also includes several code snippets ready to be copy/pasted into your own code, some reference sections for common functions and a short FAQ section for common concerns raised by users of the framework.

The tutorial comes to fill an important gap in achieving wide acceptance of atf among NetBSD developers. Some of us have been trying to migrate old regress tests to the new atf framework for a while, but such efforts are futile if other developers keep submitting new tests to the obsolete regress tree. However, such developers have been doing so because the atf documentation is not yet up to the task to guide a new developer into writing a test program for NetBSD. I expect this new tutorial to cover this gap and I hope you enjoy reading it as much as I enjoyed writing it. Comments will be highly appreciated!

Lastly, I want to give special thanks to Antti Kantee for the multiple rounds of reviews and suggestions on the text.

References

• The tutorial: Creating atf-based tests for NetBSD src
• Automated Testing Framework website

[0 comments]

 

Google Summer of Code: PostScript output for mandoc

Kristaps Dzonsons successfully implemented PostScript and PDF output for mandoc for his Google Summer of Code project.

[Read More] [1 comment]

 

Portable C Compiler

After reading about progress with the Portable C Compiler (PCC) last year, I was inspired to try building it on NetBSD. Gregory McGarry had done some work integrating it into the toolchain though it is not yet useable to build a full release, but the native build framework in external/bsd/pcc was incomplete.

[Read More] [3 comments]

 

Kernel Modules Autoload from Host in Rump

Since early 2009 NetBSD and rump has supported execution of stock kernel module binaries in userspace on x86 architectures. Starting in -current as of today, kernel modules will automatically be loaded from the host into the rump kernel. For example, when mounting a file system in a rump kernel, support will be automatically loaded before mounting is attempted.

[Read More] [3 comments]

 

NetBSD runtime linker gains negative symbol cache

The NetBSD runtime linker now has a negative symbol cache. In a nutshell, this has reduced the startup time of the Evolution mail client from around 5 minutes to 3 seconds on my QuadCore amd64 machine. Not many applications have a lot of plugins with a large amount of links to external libraries and I doubt many other applications will gain such a drastic speed bump, but the GNOME desktop as a whole now loads small bit quicker. I would imagine that KDE will now load faster as well.

[8 comments]

 

terminfo has replaced termcap

NetBSD-6 will now sport the terminfo interface which removes a lot of the problems with the old termcap which is deprecated by The Open Group. Upgrading existing systems should be quite painless as the old termcap interface is still provided, but there are some caveats.

• $TERMCAP is no longer supported, tset -s no longer exports it. So if you maintain your own terminal definition, you'll need to use tic(1) on a small terminfo database in $HOME.
• NetBSD extensions to termcap are no longer supported. Only 3rd party applications that used these would be affected.

This should allow pkgsrc not to need ncurses for a fair few console applications, like say tmux.

[0 comments]

 

NetBSD LVM enabled by default

Next release of NetBSD has received major push on storage front today, because Logical Volume Manager was enabled in current NetBSD. LVM support was committed in -current for a long time now but it was disabled by default. Today I have set MKLVM variable to yes by default which means that LVM will by included in all builds from now.

[Read More] [3 comments]

 

Summer of code results: NetBSD zfs port

This summer I worked on a port of ZFS file system to NetBSD and was mentored by Andrew Doran. This entry details the results of my Summer of Code project and future plans.

[Read More] [5 comments]