Kernel Drivers Compiled to Javascript and Run in Browser


November 07, 2012 posted by Antti Kantee

The unique anykernel capability of NetBSD allows the creation of rump kernels, which are partially paravirtualized kernels running on top of a high-level hypervisor. This technology e.g. enables running the same file system driver in the monolithic kernel or as a microkernel style server in userspace. POSIX-compatible systems have been more or less supported as rump kernel hypervisors for the past 5 years. A long-time goal has been to extend hypervisor support further, for example to embedded systems. This would bring the solid driverbase of NetBSD available to such systems with only the cost of implementing the hypervisor.

To see how far things can go, last week I started toying with the idea of using a javascript engine as a rump kernel hypervisor. I was planning to compile the NetBSD kernel sources into javascript and manually implement the hypervisor. After some searching for a C->javascript compiler, I found emscripten, which translates C into javascript via LLVM bitcode. Not only is the compiler itself extremely mature, but there is also extensive support for the POSIX API. This meant that I could not only compile the kernel drivers to javascript with emscripten, I could also compile the existing POSIX hypervisor and have it work.

The approach of compiling kernel drivers into javascript allows them to be directly accessed from existing javascript code. Yes, I did add a sys/arch/javascript into the kernel source tree. This contrasts the approach taken by another similar experiment, where an x86 Linux is run inside a x86 machine emulator running in a javascript engine.

I have thrown together a small proof-of-concept demo of how to build a web service with the capability to access file system images using kernel file system drivers compiled to javascript. I compiled a rump kernel with support for the FFS, tmpfs and kernfs file systems. This rump kernel backend is tied to a lightweight web page which passes requests from forms to the rump kernel and displays results. When the javascript is run, it downloads an FFS image (rump.data), bootstraps a rump kernel, and mounts the FFS image r/o at /ffs. The status can be further manipulated with interactive commands.

The demo is available here. I've tested it to work with Firefox and tested it to not work with Internet Explorer. YMMV with other browsers. Note, the javascript and the FFS image together are close to 5.5MB in size, so the page may load for a few moments over a slow link -- javascript is not exactly compact and whitespace removal was the only size reduction technique I used. If you're interested in comparing the generated javascript with the C sources, you can also look at the unoptimized version (14MB).

[16 comments]

 

Hydra AmigaNet / ASDG LAN Rover driver rewritten


October 30, 2012 posted by Blog Import

The ed(4) driver, supporting Hydra AmigaNet and ASDG LAN Rover cards was rewritten by Frank Wille. The new incarnation solved compatibility problems of old ed(4) driver (especially with 68040 and 68060) [0 comments]

 

End of Life for 4.x on November 17


October 27, 2012 posted by Jeff Rizzo

With the release of NetBSD 6.0, we bid a fond farewell to older releases, the NetBSD 4.x series. While we have many plans for active support of the most recent (6.x) and next most recent (5.x) release branches, 4.x is coming to the end of its supported life. As of November 17, one month after the 6.0 release, the following branches will no longer be maintained:
  • netbsd-4-0
  • netbsd-4
This means:
  • There will be no more pullups to the branches (even for security issues)
  • There will be no security advisories made for any of the 4.x releases
  • The existing 4.x releases on ftp.NetBSD.org will be moved into /pub/NetBSD-archive/

We are giving these remaining few weeks of support to give people time to migrate their machines to the newer release. We may choose to issue security advisories and/or release patches if a security issue should arise before November 17.

Here's hoping NetBSD 6 serves you as well as, or preferably better than, NetBSD 4 did!

[1 comment]

 

An X680x0 day again


October 20, 2012 posted by Blog Import

Three X68030s (including 060turbo) and X68000 PRO-II with Xellent30 running NetBSD/x68k were exhibited on NetBSD booth at Open Source Conference 2012 Hiroshima. See pictures on Togetter for details. [0 comments]

 

Proudly announcing NetBSD 6.0!


October 17, 2012 posted by Jeff Rizzo

The NetBSD Project is pleased to announce NetBSD 6.0, the fourteenth major release of the NetBSD operating system. Changes from the previous release include scalability improvements on multi-core systems, many new and updated device drivers, Xen and MIPS port improvements, and brand new features such as a new packet filter.

NetBSD 6.0 is dedicated to the memory of Allen Briggs, who passed away in March 2012.

[Read More] [11 comments]

 

(x68k) Support for network boot and formatting floppy


October 14, 2012 posted by Blog Import

Network bootloader for Neptune-X and Nereid Ethernet is added, and formatting floppy is also added. See announcement on port-x68k list for details. [0 comments]

 

Announcing Release Candidate 2 for NetBSD 6.0


September 21, 2012 posted by Jeff Rizzo

NetBSD 6.0 release candidate 2 is out!

Binaries of NetBSD 6.0_RC2 are available for download at:

ftp://ftp.netbsd.org/pub/NetBSD/NetBSD-6.0_RC2/

ISO images and (for amd64 and i386) images suitable for installing from USB sticks or other hard drives, and torrent files for downloading via BitTorrent are also available, as well as Amazon EC2 machine images and .xva files for use with Citrix XenServer and Xen Cloud Platform.

Please read the entire article for fixes

[Read More] [3 comments]

 

XM6i ver 0.38


September 15, 2012 posted by Blog Import

XM6i version 0.38 has been released. Added sound emulation, and NetBSD/amd64 6.0 binary. [0 comments]

 

Announcing Release Candidate 1 for NetBSD 6.0


August 31, 2012 posted by Jeff Rizzo

On behalf of NetBSD developers, I'm happy to announce the availability of the first release candidate of NetBSD 6.0.

Binaries of NetBSD 6.0_RC1 are available for download at:

ftp://ftp.netbsd.org/pub/NetBSD/NetBSD-6.0_RC1/

ISO images and (for amd64 and i386) images suitable for installing from USB sticks or other hard drives, and torrent files for downloading via BitTorrent are also available, as well as Amazon EC2 machine images and .xva files for use with Citrix XenServer and Xen Cloud Platform.

Fixes since NetBSD 6.0_BETA2

Thanks to tremendous testing support from the community, we are very pleased with the state of this release candidate. A number of improvements and fixes have been made since NetBSD 6.0_BETA2. Here's a sampling:

  • Fixed bugs in iscsid target handling and with large writes to iscsi devices
  • Fixed some amd64 trap-handling issues
  • Lots of fixes and improvements for npf(4)
  • Renamed wake(8) to wakeonlan(8)
  • Pass "boothowto" from the bootloader to the kernel on sparc/sparc64
  • various PUFFS/libperfuse fixes in support of glusterfs on NetBSD
  • Update BIND to 9.9.1-P2 to address CVE-2012-1667, CVE-2012-3817 and CVE-2012-3868
  • Build SQLite with SQLITE_ENABLE_COLUMN_DATA to address PR/46322
  • Fix bootstrapping the build system on some Linux distributions
  • Enable wsfb X server on vax
  • Prevent a crash when mmap()ing a block device. PR/38889
  • Sort vif and vbd device IDs on Xen so attach order is consistent
  • Make xf86-video-intel and i915drm driver work on some Atom + GMA3150 graphics chipsets
  • Update Postfix to version 2.8.11
  • Add urtw(4) and urtwn(4) wireless drivers
  • Switch device database and terminfo database to cdb(5)
  • Fixed the first-stage bootloader for Soekris net4801 PR/46027
  • Fixed a gcc bug: setjmp vs. unchanged register values PR/45837
  • Make all ttyE* entries use the wsvt25 term type PR/10127
  • Fix a bad interaction between ps(1) and ptyfs(4) PR/40813
  • Use HOST_SH instead of sh for MAKEDEV for cross builds PR/46537
  • Add support for Intel I350 Ethernet to the wm(4) driver
  • Fix several critical bugs in the x68k port
  • Remove wrong claims in md5(1) PR/44160
  • Support rc_directories in rc.shutdown PR/46601
  • Stop routed(8) from losing kernel routes PR/44647
  • Fix lack of carrier on hme(4) on some sparc64 machines PR/46424
  • Fix a panic triggered by ktrace -p PR/46591
  • Avoid failing upgrades due to spurious errors PR/12020
  • Use 32KB/4KB for default block/fragment size on >= 128 GB partitions PR/46629
  • Fix gem(4) recovery after a receiver overflow PR/46260
  • Improved support for disk "wedges"
  • Fix a crash that can happen transferring large blocks over a network. PR/43240
  • Improve linux32 compat
  • Changed root's default shell to /bin/sh
  • Fixed issues booting from CD-ROM on some i386 and amd64 systems
  • Update to tzdata2012e
  • Avoid sending bad RA packets with rtadvd(8) PR/46580
  • Fix calling setroot(9) twice PR/46732
  • Add initial support for the RaspberryPI, sufficient to boot multiuser and access the gpio header pins via serial console, but no USB or network support yet
  • Fix cc -fpie on sparc and sparc64 PR/46724
  • Don't create BSD-style pty nodes by default, since ptyfs is mounted by default.
  • Update OpenSSL to version 1.0.1c with AES-NI patch.
  • Switch off DIAGNOSTIC and/or DEBUG on release kernels
  • Fix problem with mmap and program exit PR/46807
  • Don't panic when shutting down a bridge(4) PR/46587
  • Fix locale issues in static binaries PR/46781
  • Make a number of USB memory sticks and SD card readers work which were not usable before.

Please continue to report problems and to help us test!

Known Issues

There are a small number of known issues which we expect to have fixed before the release. These fixes may require a second release candidate:

  • The OpenSSL update caused the major number of some libraries to be bumped; some software which uses these libraries may need to be recompiled from NetBSD 6.0_BETA2 and earlier, or binary packages may need to be reinstalled.
  • The vax port has a shared library problem and is not bootable in NetBSD 6.0_RC1.
  • ntpd(4) broadcast operation is not working
  • A panic can happen when an nfs rename operation fails
  • The i386 port may not boot correctly on a 486 system
  • A kernel with options DIAGNOSTICmay panic if an isp(4) card is installed
  • Xen: an amd64 DOM0 can not start X11 PR/46634

As always, please help us out by testing these changes and reporting problems either to an appropriate mailing list, via send-pr, or via the web form:

http://www.netbsd.org/cgi-bin/sendpr.cgi?gndb=netbsd

Thanks again for your help in making NetBSD 6.0 the best release yet!

[4 comments]

 

Introducing sysbuild and sysupgrade


August 07, 2012 posted by Julio Merino

If you are like me and do not like repeating manual tasks over and over again, you probably have written custom scripts to rebuild NetBSD from source and to upgrade your NetBSD installation to a new build. I have had such scripts for years, but they have always been very rudimentary, inflexible and unreliable. What's more: every NetBSD machine I have today has its own custom versions of these scripts, often with little incompatible tweaks to accomodate every installation.

I was now facing the time to rebuild my NetBSD development box (because its VM disk is running out of space), and the thought of having to reconfigure my environment to perform periodic NetBSD builds, and to have to continue using my deficient custom upgrade script gave me the energy to sit down and write decent versions of these two tools. The results of such efforts are sysbuild and sysupgrade, and their goal is to trivialize rebuilds and upgrades.

sysbuild, currently available in pkgsrc/sysutils/sysbuild, automates the process of fetching the NetBSD sources from a CVS repository and performing a build for one or more platforms. The process is defined by a configuration file that controls the parameters of the build, and building a new snapshot from scratch is as easy as issuing one parameter-less command. A companion sysbuild-user package configures an unprivileged system user to perform nightly builds of NetBSD from cron, which is very convenient for developers tracking NetBSD-current. My other post "Introducing sysbuild for NetBSD" and the sysbuild(1) manual page provide more details.

sysupgrade, currently available in pkgsrc/sysutils/sysupgrade, automates the process of bringing a NetBSD system up to date by using a collection of distribution sets (which can come from either sysbuild, your manual builds or a public FTP site). This tool does not have any upgrade knowledge of its own: it is "just" a script for the manual procedure of fetching the distribution sets, upgrading the kernel, unpacking the sets and upgrading /etc with the standard etcupdate and postinstall. My other post "Introducing sysupgrade for NetBSD" and the sysupgrade(8) manual page provide more details.

The tools are released under a typical permissive 3-clause BSD license. I have opted to place them in pkgsrc instead of the base system to give the tools some mileage and to make them immediately available to older releases and operating systems. However, my end goal for these tools (specially sysupgrade) is to end up being part of the NetBSD base system: I find it unfortunate that the base system does not provide a mechanism to upgrade itself to a newer release.

Enjoy the tools and please share your feedback!

... just keep in mind that these are fresh new code so obscure bugs may be lingering around. Hint: Run the included test suites to get some confidence in the implementation before running the tools.

[1 comment]

 

Wanna Raspberry?


July 30, 2012 posted by Mike M. Volokhov

Demand for a cheap yet powerful computer is pretty high in geek circles. Remember those days when you could take a ZX Spectrum, or Commodore 64, or something like that and attach it to your TV to fill next few days with a great fun. Since then, PCs pushed out these tiny home computers. But today, Raspberry Pi — a little credit-card sized sweet computer — is here. Nick's Raspberry Pi

The Raspberry Pi is built on Broadcom BCM2835 system on chip which contains ARM1176 core running at 700MHz, with VideoCore 4 GPU, and has 256 MB of RAM on board. It provides two USB ports, 100Mb/s Ethernet, HDMI and composite video; audio output presented as well. Operating system loaded from SD flash.

Yet more fun is that the initial NetBSD support was just committed to the NetBSD source tree, and the Raspberry Pi can boot into multiuser now. Currently work on device drivers is in progress. USB and Ethernet are planned to be supported next.

Generally, NetBSD support for the ARM1176 core is in very good shape. The largest parts of the code I committed were the update to the plcom serial driver and the infrastructure to support the low level parts of the bcm2835 (interrupt controller, bus_space, and timer) — says Nick Hudson, who did the bringup. However, the porting has some hard nuts to crack: The graphics part is a bit of a challenge. But I hope to get dumb framebuffer support relatively soon. There is a publicly available datasheet for part of bcm2835, but certainly not the video controller.

Matt Thomas, the port-evbarm portmaster, was very knowledgeable and helpful in answering all my questions. I'd like to also thank Stephen Borrill for getting me an RPI in the first place. Stephen spoke to Eben Upton in Cambridge and soon after an RPI arrived — adds Nick.

So, there is a boot log from the very first NetBSD driven RPI board, and work on the device is in full swing. Many people exposed interest, and eventually the Raspberry Pi could became a good alternative for number of well supported, but aging single board computers.

[23 comments]

 

G-REX PCI driver updated


July 11, 2012 posted by Blog Import

The G-REX bridge driver was updated and now supports all available PCI slots. Functionality of G-REX is now on a par with Mediator 1200 in the NetBSD. [0 comments]