NetBSD 6.1 Release Candidate 4

The fourth release candidate of NetBSD 6.1 is now available for download at: http://ftp.NetBSD.org/pub/NetBSD/NetBSD-6.1_RC4/. It is expected that this will be the final release candidate, with the official release following very soon.

(Please note that while the third release candidate (RC3) was tagged and built, it was never officially released)

NetBSD 6.1 will be the first feature update for the NetBSD 6 branch. There are many new drivers, some new features, and many bug fixes! Fixes since RC2 include:

• Updated the fix for SA-2013-003 (RNG bug may result in weak cryptographic keys)
• Fixes to npfctl(8) parsing and error handling
• Fix sendto(2) issue with IPv6 UDP datagrams. PR#47408.
• Raspberry Pi: fix handling of large packets in USB host controller
• Fixed an RPC memory corruption issue. PR#13082
• Fixed ACPI issues affecting some AMD systems. PR#47016, PR#47648.
• Change vax MAXPARTITION from 16 to 12, addressing boot issues on some systems
• Bump libpthread minor version to libpthread.so.1.1 for the addition of pthread_cond_setclock() earlier in the 6.1 release cycle; note that this is *NOT* the same as libpthread.so.1.1 in NetBSD-current. (libpthread.so in NetBSD-current is already at version 1.2)
• Provide libc stubs to libpthread, allowing libpthread to be dlopen()ed.
• Fix a userland-triggered panic on x68k systems.

A complete list of changes can be found at:
http://ftp.NetBSD.org/pub/NetBSD/NetBSD-6.1_RC4/CHANGES-6.1

Please help us test this and any upcoming release candidates as much as possible. Remember, any feedback is good feedback. We'd love to hear from you, whether you've got a complaint or a compliment.

[1 comment]

 

Thylacine USB driver

Zorro attachment code for slhci(4) driver was added. Thylacine USB card is now supported. However, so far only keyboards and mice work reliably. To use them instead of standard Amiga keyboard and mouse wscons kernel is required. [0 comments]

 

XM6i ver 0.41

XM6i version 0.41 has been released. Now XM6i supports MC68881 FPU emulation and the performance of NetBSD/x68k on XM6i is notably improved. XM6i's FPU emulation code was based on NetBSD/m68k FPE sources, and then the XM6i developments also improve NetBSD/m68k FPE implementation. [0 comments]

 

A Rump Kernel Hypervisor for the Linux Kernel

Ever since I realized that the anykernel was the best way to construct a modern general purpose operating system kernel, I have been performing experiments by running unmodified NetBSD kernel drivers in rump kernels in various environments (nb. here driver does not mean a hardware device driver, but any driver like a file system driver or TCP driver). These experiments have included userspaces of various platforms, binary kernel modules on Linux and others, and compiling kernel drivers to javascript and running them natively in a web browser. I have also claimed that the anykernel allows harnessing drivers from a general purpose OS onto more specialized embedded computing devices which are becoming the new norm. This is an attractive possibility because while writing drivers is easy, making them handle all the abnormal conditions of the real world is a time-consuming process. Since the above-mentioned experiments were done on POSIX platforms (yes, even the javascript one), the experiments did not fully support the claim. The most interesting, decidedly non-POSIX platform I could think of for experimentation was the Linux kernel. Even though it had been several years since I last worked in the Linux kernel, my hypothesis was that it would be easy and fast to get unmodified NetBSD kernel drivers running in the Linux kernel as rump kernels.

A rump kernel runs on top of the rump kernel hypervisor. The hypervisor provides high level interfaces to host features, such as memory allocation and thread creation. In this case, the Linux kernel is the host. In principle, there are three steps in getting a rump kernel to run in a given environment. In reality, I prefer a more iterative approach, but the development can be divided into three steps all the same.

1. implement generic rump kernel hypercalls, such as memory allocation, thread creation and synchronization
2. figure out how to compile and run the rump kernel plus hypervisor in the target environment
3. implement I/O related hypercalls for whatever I/O you plan to do

Getting basic functionality up and running was a relatively straightforward process. The only issue that required some thinking was an application binary interface (ABI) mismatch. I was testing on x86 where Linux kernel ABI uses -mregparm=3, which means that function arguments are passed in registers where possible. NetBSD always passes arguments on the stack. When two ABIs collide, the code may run, but since function arguments passed between the two ABIs result in garbage, eventually an error will be hit perhaps in the form of accessing invalid memory. The C code was easy enough to "fix" by applying the appropriate compiler flags. In addition to C code, a rump kernel uses a handful of assembly routines from NetBSD, mostly pertaining to optimizations (e.g. ffs()), but also to access the atomic memory operations of the platform. After assembly routines had been handled, it was possible to load a Linux kernel module which bootstraps a rump kernel in the Linux kernel and does some file system operations on the fictional kernfs file system. A screenshot of the resulting dmesg output is shown below.

It is one thing to execute a computation and an entirely different thing to perform I/O. To test I/O capabilities, I ran a rump kernel providing a TCP/IP driver inside the Linux kernel. For a networking stack to be able to do anything sensible, the interface layer needs to be able to shuffle packets. The quickest way to implement the hypercalls for packet shuffling was to use the same method as a userspace virtual TCP/IP stack might use: read/write packets using the tap device. Some might say that doing this from inside the kernel is cheating, but given that the alternative was to copypaste the tuntap driver and edit it slightly, I call my approach constructive laziness.

The demo itself opens a TCP socket to port 80 on vger.kernel.org (IP address 0x43b484d1 if you want to be really precise), does a HTTP get for "/" and displays the last 500 bytes of the result. TCP/IP is handled by the rump kernel, not by the Linux kernel. Think of it as the Linux kernel having two alternative TCP/IP stacks. Again, a screenshot of the resulting dmesg is shown below. Note that unlike in the first screenshot, there is no printout for the root file system because the configuration used here does not include any file system support. Yes, you can ping 10.0.2.17.

As hypothesized, a rump kernel hypervisor for the Linux kernel was easy and straightforward to implement. Furthermore, it could be done without making any changes to the existing hypercall interface thereby reinforcing the belief that unmodified NetBSD kernel drivers can run on top of most any embedded firmwares just by implementing a light hypervisor layer.

There were no challenges in the experiment, only annoyances. As Linux does not support rump kernels, I had to revert back to the archaic full OS approach to kernel development. The drawbacks of the full OS approach include for example suffering multi-second reboot cycles during iterative development. The other tangential issue that I spent a disproportionately large amount of time with was thinking about how releasing this code would affect existing NetBSD code due to GPL involvement. My conclusion was that this does not matter since all code used by the current demo is open source anyway, and if someone wants to use my code in a product, it is their problem, not mine.

For people interested in examining the implementation, I put the source code for the hypervisor along with the test code in a git repo here. The repository also contains the demos linked from this article. The NetBSD kernel drivers I used are available from ftp.netbsd.org or by getting buildrump.sh and running ./buildrump.sh checkout.

[2 comments]

 

pointers to rpi docs 2013Q1

We get a lot of comments asking for tips on using the raspberry pi so I thought I would point out some docs:
evbarm/rpi wiki docs
An example of the rpi.img can be found here:
http://nyftp.netbsd.org/pub/NetBSD-daily/HEAD/201303221130Z/evbarm/binary/gzimg/ notice the HEAD (NetBSD -current), datestamp, arch path for future reference

There are also some concerns about building a kernel/img on your own.
building NetBSD
build.sh is one of the best features of NetBSD. You can cross compile from almost any other unix-like system with very little difficulty.

[1 comment]

 

NetBSD 6.1 Release Candidate 2

The second release candidate of NetBSD 6.1 is now available for download at: http://ftp.NetBSD.org/pub/NetBSD/NetBSD-6.1_RC2/

NetBSD 6.1 will be the first feature update for the NetBSD 6 branch. There are many new drivers, some new features, and many bug fixes! Fixes since RC1 include:

• Various terminfo fixes (PR#46793, PR#47090, PR#47490, PR#47532)
• Fixed a segfault in awk(1) (PR#47553)
• Moved boottime50 and its associated sysctl into the compat module. (PR#47579)
• Updated tzdata to 2013b, with the latest timezone info
• Fixed a crash when the security.curtain sysctl is enabled (PR#47598)
• Fixed some IPF locking issues
• Fix a crash on statically-linked programs for NetBSD/alpha

A complete list of changes can be found at:
http://ftp.NetBSD.org/pub/NetBSD/NetBSD-6.1_RC2/CHANGES-6.1

Please help us test this and any upcoming release candidates as much as possible. Remember, any feedback is good feedback. We'd love to hear from you, whether you've got a complaint or a compliment.

[1 comment]

 

Initial Kyua import done

I am very happy to announce that the initial import of Kyua into the NetBSD source tree is complete! I am sure there are some rough edges and here is where you come into play.

First of all, let me clarify that all the integration changes are gated by the MKKYUA build setting, which still defaults to no. Unless you explicitly set MKKYUA=yes in /etc/mk.conf or in the build.sh command line, you will not get a system with Kyua. However, once you set that flag, you will transition to the full new setup:

• Kyua will be installed as /usr/bin/kyua. See kyua(1) to get started.
• The old atf-run and atf-report tools will become compatibility wrappers. These should be a reasonable drop-in replacement for most use cases, but they are probably not perfect.
• Your /usr/tests tree will be populated by a bunch of Kyuafiles.

So what's new compared to the ATF tools? Here are some highlights:

• Ability to write and run ATF-less tests. It has been a common desire around here to develop test programs that do not rely on the ATF libraries. With this change, this becomes possible. See kyuafile(5) for details.
• Direct HTML report generation. There is no need to set up a complex XML toolchain any more to convert the ATF test reports into HTML pages. Kyua can generate plain HTML directly, so it will be feasible to serve such content straight from NetBSD's built-in httpd.
• Historical data. As seen in the various test beds that have appeared around ATF, there is a desire to maintain historical data of the test results. Kyua does that natively, by recording the results of the execution in a SQLite database. Reports can later be extracted from this database. There is still a lot of room for improvement here.
• More flexible metadata and configuration. While this does not provide a real advantage today, as soon as the old atf-run and atf-report tools are gone we can trivially fix some long-standing issues (e.g. the inability to customize test deadlines).
• Less complexity during test case execution. As you may have noticed over the years, the code in atf-run to capture the output of tests and deal with interrupts is not particularly robust. There have been several problems in this area, and I'm not convinced that they are all fixed. The new code works in a different manner and has been more carefully thought around these edges.
• Independent testers. The code that implements the isolation of test cases and their controlled execution has been split into a set of "testers" that live in /usr/libexec/kyua-*-tester. These tools provide scriptable interfaces to interact with tests, with the idea that the kyua(1) frontend should end up being pretty straightforward. Should you want to write your own trivial script to run tests without kyua(1), you could pretty easily do that by interacting with the testers directly.

How can you help? Easy. Just rebuild your system with MKKYUA=yes, read through kyua(1), start using it to stress-test your system and report any problems you may encounter!

My immediate next steps include addressing your feedback and working with our major test runners to add support to their systems to use the new tools (for example, change anita to support running tests with kyua(1)).

Enjoy and thanks for reading. [0 comments]

 

NetBSD 6.1 Release Candidate 1

The first release candidate of NetBSD 6.1 is now available for download at: http://ftp.NetBSD.org/pub/NetBSD/NetBSD-6.1_RC1/

NetBSD 6.1 will be the first feature update for the NetBSD 6 branch. There are many new drivers, some new features, and many bug fixes! Highlights include:

• Bugfixes and feature improvements to NPF, the NetBSD Packet Filter
• Improvements to several ARM platforms, including Raspberry Pi which now has nearly-complete support.
• Support for dtrace on amd64
• MIPS ports switched to gdb 7.3.1, gdb6 removed
• Additional device support in key drivers including wm(4), uftdi(4), mfi(4), bge(4), aac(4), tlp(4) and others.
• Various port-specific improvements to the amiga, arm, sparc64 and x68k ports.

A complete list of changes can be found at:
http://ftp.NetBSD.org/pub/NetBSD/NetBSD-6.1_RC1/CHANGES-6.1

Please help us test this and any upcoming release candidates as much as possible. Remember, any feedback is good feedback. We'd love to hear from you, whether you've got a complaint or a compliment.

[6 comments]

 

Mediator 4000 now supported

Family of ELBOX Mediator 4000 PCI bridges is now supported by the new em4k(4) driver. It was tested successfully with SATALink 3114 and NE2000-compatible ethernet cards. Work on graphics cards support is in progress. [0 comments]

 

Software controlled power-off

The new empm(4) driver adds support for power management circuit on ELBOX Mediator 1200 TX and SX models. It allows powering off the Amiga from software if ATX PSU is used. [0 comments]

 

(luna68k) Native bootloader

Native bootloader support for NetBSD/luna68k ELF kernel has been committed. It's based on 4.4BSD/luna68k's "Stinger" bootloader written back in 1992 and 1993, and now we can load NetBSD/luna68k kernels from any SCSI disks or over LANCE Ethernet both on LUNA-I and LUNA-II. [0 comments]

 

NetBSD 6.0.1 released

The NetBSD Project is pleased to announce NetBSD 6.0.1, the first security/bugfix update of the NetBSD 6.0 release branch. It represents a selected subset of fixes deemed important for security or stability reasons.

For more details, please see the 6.0.1 release notes

Complete source and binaries for NetBSD 6.0.1 are available for download at many sites around the world. A list of download sites providing FTP, AnonCVS, SUP, and other services may be found at http://www.NetBSD.org/mirrors/. [0 comments]