BSDCan 2009: Kernel Development in Userspace - The Rump Approach

At BSDCan 2009 I gave a presentation about using rump for kernel development. The associated paper has two target audiences:

• regular users who wish to easily learn about how the kernel works without having to setup complex kernel development environments
• kernel developers who wish to learn how to use rump for kernel development and requesting more detailed information from users in problem reports

Downloads:

• paper
• slides

[0 comments]

 

pkgin, a tool to manage pkgsrc binary packages

From the day I began using NetBSD I felt that there was a need for a binary package manager. pkgsrc is great of course, I use it, love it and contribute to, but I'm not brave enough to build my entire environment with it. Of course the esteemed pkg_add(1) and pkg_delete(1) can handle binary packages installation, but when it comes to upgrades, binary packages manipulation is far from being straightforward.

[Read More] [0 comments]

 

netpgp

NetBSD-current has recently had a new addition - that of netpgp, a BSD-licensed library that can perform digital signature signing and verification, and also encryption and decryption of files. An email explaining netpgp was sent to the tech-security mailing list, but I thought I'd give a short summary of that email here...

[Read More] [0 comments]

 

Runnable Userspace Meta Programs in NetBSD 5.0

The rump (Runnable Userspace Meta Program) framework, unique to NetBSD, provides lightweight virtualization of kernel components such a file systems and networking. This short article explores the key ideas behind rump and gives examples on how to use rump on NetBSD 5.

[Read More] [0 comments]

 

Impromptu bugathon

Yesterday, prompted by the threat of reaching 5000 active bug reports several developers began fixing and closing.

As of this writing the count of active PRs has dropped from 4976 to 4893 -- that's 83 resolved -- and quite a few more have been placed in submitter feedback awaiting confirmation that they are in fact fixed.

This is a positive step, and a pretty big number for a single day that wasn't a previously scheduled bugathon. Unfortunately, to clear through the backlog we'd have to do this much every single day for two full months.

Despite all the cleanup work over the past year and a half, the database is still full of reports of already-fixed problems. These not only inflate the total counts; they get in the way and make it harder to find real problems to work on. Finding and closing them needs to be a group effort.

If you're a developer: when you fix a problem please include the PR number in the commit message so it gets pulled into the database; also, please close PRs that you fix, or leave a comment explaining what still needs to be done (e.g. pullups, real fixes vs. quick hacks, etc.)

If you're a user: if you have filed a bug report and it's been fixed, but the report is still open, please follow up to let us know it can be closed. And also, note that GNATS is particular about followups: they need to be sent to gnats-bugs AT netbsd.org (not gnats-admin) and the Subject: line needs to begin with "Re:" and the category and bug number in the form "port-eniac/12345". Other followups are likely to be dropped as spam, or may end up opening a new PR instead. In any event, if you send a followup and you don't get a copy back from GNATS, it probably got lost. If you can't figure out what's wrong, post to the netbsd-bugs mailing list.

And finally, if you are tired of the Gnats search interface you may find these lists helpful... but be aware that they often don't update very rapidly as there are manual steps involved.

[0 comments]

 

Bluetooth stack/sdp protocol update

NetBSD bluetooth stack was updated to properly support Service discovery protocol spec. SDP daemon sdpd(8) was rewritten to be much simpler, and finally updated the sdpquery(1) program to display complete service records in a meaningful manner.

[Read More] [0 comments]

 

EuroBSDCon 2009 - Call for Papers

EuroBSDCon 2009 - Call for Papers
9th European BSD Conference
September 18 - 20, 2009
University of Cambridge, UK
http://2009.eurobsdcon.org/

Introduction

The European BSD Community is once again gathering for EuroBSDcon. In 2009, we invite you to join us in Cambridge, England for the latest in discussion, dissemination and development of material from the many BSDs and their related communities. This, the ninth European BSD conference is a great opportunity to present new ideas to the community and to meet some of the developers behind the different BSDs. The two day conference program (September 19 - 20) will be complemented by a tutorial day preceding the conference (Sept 18).

Call for Papers

The Conference is inviting authors to submit innovative and original papers not submitted to other European conferences on the applications, architecture, implementation, performance and security of BSD-derived operating systems. Investigations on economic aspects regarding the operation of BSD systems are also welcome. Topics of interest for the EuroBSD Conference 2009 include, but are not limited to:

• application development and deployment
• device drivers
• security and safe coding practices
• methods others should know about
• system administration: techniques and tools of the trade
• operational and economic aspects

Prospective authors of contributions to the technical program are requested to submit an abstract via http://2009.eurobsdcon.org/. All submissions will be acknowledged. Presentations may last from 15 to 45 minutes - please indicate how long you would like. This is the initial call for papers; a more focussed call based on initial accepted submissions will follow in March 2009. We will begin accepting talks early in 2009. Authors of accepted submissions should provide a full paper for publication in the conference proceedings and give permission to the organizers to publish the results in the printed proceedings and on the conference web site at www.eurobsdcon.org

Call for Tutorial Proposals

Selected tutorials on practical and problem-solving aspects of BSD-derived operating systems will be offered on the day before the Conference. The tutorials will be presented by speakers who have wide experience in developing and administering the different BSDs. Potential tutorial themes could include, but are not limited to:

• Safe coding practices to provide secure solutions
• System load testing and tuning
• BSD in a large network
• Solving sets of problems

If you are interested in presenting a tutorial, please contact the organisers on eurobsdcon@ukuug.org with what you're thinking. Initial exploratory conversations are as welcome as full proposals.

Sponsorship Opportunities

We are seeking companies or institutions to sponsor various elements of the conference in order to keep delegate fees as low as possible. Sponsorship opportunities include: paying for a speaker's travel or accommodation; providing bursaries for delegates who cannot pay the conference fee themselves; sponsoring catering, lunches, or the conference dinner. All sponsors will be listed in the conference proceedings and included on our website with a link back to your site. You will also have the opportunity to provide literature for distribution in delegate packs. Please contact the UKUUG Secretariat (office@ukuug.org) to discuss the possibilities or see http://www.eurobsdcon.org/.

Important Dates

Final abstract deadline:	May 31st 2009
Final tutorial deadline:	May 31st
Final papers due:	August 1st
Tutorial day:	September 18
Conference:	September 19 - 20

For more, see http://www.eurobsdcon.org/.

[0 comments]

 

NetBSD 5.0: benchmarks and an introduction

I have prepared a presentation giving an overview of some of the new features, technology and performance improvements debuting in NetBSD 5.0. You can find the slides from this presentation here:

• HTML format (browseable)
• PDF format (for printers or projectors)

A comparison of the scalability and performance improvements found in NetBSD 5.0 was conducted against NetBSD 4.0, Fedora Core 10 and FreeBSD 7.1, and is part of the presentation. An outline is below, but for full details on the tests, please refer to the slides. Please click on the images to view a larger version of each.

Benchmark: hackbench

This benchmark tests the efficiency and scalability of the scheduler and IPC (inter-process commmunication) mechanisms. It was created by and is popular with Linux kernel developers. Hackbench quickly exposes problems with SMP scaling, and with high numbers of active tasks.

Benchmark: MySQL sysbench

This test simulates an OLTP (OnLine Transaction Processing) style database workload. From the OS developer's viewpoint, it measures the efficiency and scalability of threading, memory allocation, and IPC.

Benchmark: build.sh, NetBSD's build system

This test measures how well the operating system scales when given hosting an intensive software development or shell type workload.

Fedora continues to excel due to the significant investment in improving the efficiency and scalability of Linux over the last few years. Our hats are off to the GNU/Linux/Fedora developers. We aim to reverse the position with the release of NetBSD 6.0.

[2 comments]

 

Automated testing in NetBSD 5.0

With the release of NetBSD 5.0, a new testing infrastructure for the operating system will get wide exposure. This testing infrastructure is based on the Automated Testing Framework (ATF), a project that was started as part of the Google Summer of Code 2007 program, and that provides a platform-independent framework to easily write and automatically exercise test cases. As of this release, only a few NetBSD-specific tests are available as ATF-based tests, but they are a good preview of what the future will look like.

Starting from this release, you will see a new tests.tgz distribution set during the installation of NetBSD. If you choose to install it, sysinst will populate /usr/tests with a new collection of test programs that are based on the ATF framework. Once installed, edit /etc/atf/NetBSD.conf to suit your system preferences and then, to run the tests, do:

# cd /usr/tests
# atf-run | atf-report

The whole idea of providing the test programs as an installable distribution set is that your specific combination of hardware and software is not available to system developers, so only you can make sure that the system behaves as it should. Furthermore, by successfully running the tests, you can have a good feeling that everything is working as expected!

As far as I know, there are some tests in this release that are broken, but I'm not sure if this is because the tests themselves are broken or because the features under test are broken. Help is welcome in this area!

There is still a lot of work to do in the automated testing area... but the fact that ATF is now bundled in a formal release of NetBSD raises my willingness to work on it. If only I had enough time... At the very least, expect many more ATF-based test cases in NetBSD 6.0.

[0 comments]

 

Announcing NetBSD 5.0

On behalf of the NetBSD developers, I am proud to announce that NetBSD 5.0, the thirteenth release of the NetBSD operating system, is now available.

NetBSD 5.0 features greatly improved performance and scalability on modern multiprocessor (SMP) and multi-core systems. Multi-threaded applications can now efficiently make use of more than one CPU or core, and system performance is much better under I/O and network load.

This improved performance is the result of a rewritten threading subsystem based on a 1:1 threading model, new kernel synchronization primitives, kernel preemption, a rewritten scheduler implementation, real-time scheduling extensions, processor sets, and dynamic CPU sets for thread affinity. Almost all core kernel subsystems, like virtual memory, memory allocators, file system frameworks for major file systems, and others were audited and overhauled to make use of highly concurrent algorithms.

In addition to scalability and performance improvements, a significant number of major features have been added. Some highlights are: a preview of metadata journaling for FFS file systems (known as WAPBL, Write Ahead Physical Block Logging), the 'jemalloc' memory allocator, the X.Org X11 distribution instead of XFree86 on a number of ports, the Power Management Framework, ACPI suspend/resume support on many laptops, write support for UDF file systems, the Automated Testing Framework, the Runnable Userspace Meta Program framework, Xen 3.3 support for both i386 and amd64, POSIX message queues and asynchronous I/O, and many new hardware device drivers.

For full details, please see the 5.0 release notes.

Complete source and binaries for NetBSD 5.0 are available for download at many sites around the world. A list of download sites providing FTP, AnonCVS, and other services may be found at http://www.NetBSD.org/mirrors/

[0 comments]

 

dhcpcd-5 released and imported into NetBSD -current

dhcpcd-5 has now been imported into -current. I've run a full distribution build and install without any issues so hopefully nothing broken :)

Here's a list of the major changes from dhcpcd-4

• Single daemon can now run DHCP on multiple interfaces at the same time
• Configuration profiles per interface, ssid, arping and fallback
• Listens to 3rd party programs changing routing information
• Supports DHCP INFORM over PPP
• Can configure static options for destination address
• Control socket so 3rd party program can control or listen directly to dhcpcd events
• Is also a BOOTP client

There's now an rc.d script for it which is now recommended over ifconfig_bge0=dhcp in /etc/rc.conf

Any dhcpcd-gtk users will need to upgrade to dhcpcd-gtk-0.4.0 as there was a last minute variable rename due to a recent rc.subr change looking for the flags environment variable.

Enjoy!

[0 comments]

 

Summer of Code 2009 is on!

For the fifth consecutive year, the NetBSD Project is proud to participate in Google's Summer of Code program as a mentoring organization and we're pleased to announce the list of projects that have been accepted for this summer. This year's selected students include a number of NetBSD developers, returning SoC alumni and a few freshmen. We're very excited to have projects ranging from the areas of filesystems over install automation to userland tools and we expect the entire NetBSD community to benefit tremendously.

In the coming weeks, you will see our students engage the NetBSD community for support with their projects; please give them a warm welcome and help our developers, students and mentors lead all these projects to success!

[0 comments]