GSoC Reports: Curses Library Automated Testing, Part 1

This report was prepared by Naman Jain as a part of Google Summer of Code 2020

Introduction

My GSoC project under NetBSD involves the development of test framework of curses library. Automated Test Framework (ATF) was introduced in 2007 but ATF cannot be used directly for curses testing for several reasons most important of them being curses has functions which do timed reads/writes which is hard to do with just piping characters to test applications. Also, stdin is not a tty device and behaves differently and may affect the results. A lot of work regarding this has been done and we have a separate test framework in place for testing curses.

The aim of project is to build a robust test suite for the library and complete the SUSv2 specification. This includes writing tests for the remaining functions and enhancing the existing ones. Meanwhile, the support for complex character function has to be completed along with fixing some bugs, adding features and improving the test framework.

[Read More] [0 comments]

 

GSoC Reports: Fuzzing the NetBSD Network Stack in a Rumpkernel Environment, Part 1

This report was prepared by Nisarg Joshi as a part of Google Summer of Code 2020

Introduction:

The objective of this project is to fuzz the various protocols and layers of the network stack of NetBSD using rumpkernel. This project is being carried out as a part of GSoC 2020. This blog post is regarding the project, the concepts and tools involved, the objectives and the current progress and next steps.

Fuzzing:

Fuzzing or fuzz testing is an automated software testing technique in which a program is tested by passing unusual, unexpected or semi-random input generated data to the input of the program and repeatedly doing so, trying to crash the program and detect potential bugs or undealt corner cases.

There are several tools available today that enable this which are known as fuzzers. An effective fuzzer generates semi-valid inputs that are "valid enough" in that they are not directly rejected by the parser, but do create unexpected behaviors deeper in the program and are "invalid enough" to expose corner cases that have not been properly dealt with. 

Fuzzers can be of various types like dumb vs smart, generation-based vs mutation-based and so on. A dumb fuzzer generates random input without looking at the input format or model but it can follow some sophisticated algorithms like in AFL, though considered a dumb fuzzer as it just flips bits and replaces bytes, still uses a genetic algorithm to create new test cases, where as a smart fuzzer will follow an input model to generate semi-random data that can penetrate well in the code and trigger more edge cases. Mutation and generation fuzzers handle test case generation differently. Mutation fuzzers mutate a supplied seed input object, while generation fuzzers generate new test cases from a supplied model.

Some examples of popular fuzzers are: AFL(American Fuzzy Lop), Syzkaller, Honggfuzz.

[Read More] [0 comments]

 

GSoC Reports: Make system(3) and popen(3) use posix_spawn(3) internally, Part 1

This report was prepared by Nikita Ronja Gillmann as a part of Google Summer of Code 2020

This is my first report for the Google Summer of Code project I am working on for NetBSD.

Prior work: In GSoC 2012 Charles Zhang added the posix_spawn syscall which according to its SF repository at the time (maybe even now, I have not looked very much into comparing all other systems and libcs + kernels) is an in-kernel implementation of posix_spawn which provides performance benefits compared to FreeBSD and other systems which had a userspace implementation (in 2012).

After 1 week of reading POSIX and writing code, 2 weeks of coding and another 1.5 weeks of bugfixes I have successfully implemented posix_spawn in usage in system(3) and popen(3) internally.

The biggest challenge for me was to understand POSIX, to read the standard. I am used to reading more formal books, but I can't remember working with the posix standard directly before.

The next part of my Google Summer of Code project will focus on similar rewrites of NetBSD's sh(1).

[Read More] [0 comments]

 

GSoC Reports: Fuzzing Rumpkernel Syscalls, Part 1

This report was prepared by Aditya Vardhan Padala as a part of Google Summer of Code 2020

It has been a great opportunity to contribute to NetBSD as a part of Google Summer Of Code '20. The aim of the project I am working on is to setup a proper environment to fuzz the rumpkernel syscalls. This is the first report on the progress made so far.

Rumpkernels provide all the necessary components to run applications on baremetal without the necessity of an operating system. Simply put it is way to run kernel code in user space.

The main goal of rumpkernels in netbsd is to run,debug,examine and develop kernel drivers as easy as possible in the user space without having to run the entire kernel but run the exact same kernel code in userspace. This makes most of the components(drivers) easily portable to different environments.

Rump Kernels are constructed out of components, So the drivers are built as libraries and these libraries are linked to an interface(some application) that makes use of the libraries(drivers). So we need not build the entire monolithic kernel just the required parts of the kernel.

[Read More] [0 comments]

 

VAX port needs help

Looking for volunteers to help VAX gcc, now collecting bounties...

[Read More] [5 comments]

 

Announcing Google Summer of Code 2020 projects

We are very happy to announce The NetBSD Foundation Google Summer of Code 2020 projects:

• Apurva Nandan - Benchmark NetBSD
• Jain Naman - Curses library automated testing
• Nikita Gillmann - Make system(3) and popen(3) use posix_spawn(3) internally
• Ayushi Sharma - Enhance the syzkaller support for NetBSD
• Aditya Vardhan Padala - Rumpkernel Syscall Fuzzing
• Nisarg Joshi - Fuzzing the network stack of NetBSD in a rumpkernel environment
• Jason High - Extending the functionality of the netpgp suite

The community bonding period - where students get in touch with mentors and community - started on May 4 and will go on until June 1. The coding period will be June 1 to August 24.

Please welcome all our students and a big good luck to students and mentors!

A big thank you to Google and The NetBSD Foundation organization mentors and administrators!

Looking forward to having another nice Google Summer of Code!

[2 comments]

 

The GNU GDB Debugger and NetBSD (Part 2)

The NetBSD team of developers maintains two copies of GDB:

• One in the base-system with a stack of local patches.
• One in pkgsrc with mostly build fix patches.

The base-system version of GDB (GPLv3) still relies on a set of local patches. I set a goal to reduce the local patches to bare minimum, ideally reaching no local modifications at all.

Over the past month I've reimplemented debugging support for multi-threaded programs and upstreamed the support. It's interesting to note that the old support relied on GDB tracking only a single inferior process. This caused the need to reimplement the support and be agnostic to the number of traced processes. Meanwhile the upstream developers introduced new features for multi-target tracing and a lot of preexisting code broke and needed resurrection. This affected also the code kept in the GDB basesystem version. Additionally over the past 30 days, I've also developed new CPU-independent GDB features that were for a long time on a TODO list for NetBSD.

After the past month NetBSD has now a decent and functional GDB support in the mainline. It's still not as featured as it could and CPU-specific handling will need a dedicated treatment.[Read More] [0 comments]

 

Improving libossaudio, and the future of OSS in NetBSD

Nia discusses recent fixes she's made to the Open Sound System compatibility layer, and explains some of the history behind OSS, and its future in NetBSD.

[Read More] [1 comment]

 

Wifi renewal restarted

I have started work on the phil-wifi branch, trying to modernize our net80211 and sync it with FreeBSD.

[Read More] [1 comment]

 

LLDB work concluded

Upstream describes LLDB as a next generation, high-performance debugger. It is built on top of LLVM/Clang toolchain, and features great integration with it. At the moment, it primarily supports debugging C, C++ and ObjC code, and there is interest in extending it to more languages.

In February 2019, I have started working on LLDB, as contracted by the NetBSD Foundation. So far I've been working on reenabling continuous integration, squashing bugs, improving NetBSD core file support, extending NetBSD's ptrace interface to cover more register types and fix compat32 issues, fixing watchpoint and threading support, porting to i386.

March 2020 was the last month of my contract. During it my primary focus was to prepare integration of LLDB into NetBSD's src tree.

[Read More] [0 comments]

 

The GNU GDB Debugger and NetBSD (Part 1)

The NetBSD team of developers maintains two copies of GDB:

• One in the base-system with a stack of local patches.
• One in pkgsrc with mostly build fix patches.

The process of maintaining a modern version (GPLv3) of GDB in basesystem is tainted with a constant extra cost. The NetBSD developers need to rebase the stack of local patches for the newer releases of the debugger and resurrect the support. The GDB project is under an active development and in active refactoring of the code, that was originally written in C, to C++.

Unfortunately we cannot abandon the local basesystem patches and rely on a pristine version as there is lack of feature parity in the pkgsrc version of GDB: no threading support, not operational support for most targets, no fork/vfork/etc events support, no auxv reading support on 64-bit kernels, no proper support of signals, single step etc.

Additionally there are extra GDB patches stored in pkgsrc-wip (created by me last year), that implement the gdbserver support for NetBSD/amd64. gdbserver is a GDB version that makes it possible to remotely debug other programs even across different Operating Systems and CPUs. This code has still not been merged into the mainline base-system version. This month, I have discovered that support needs to be reworked, as the preexisting source code directory hierarchy was rearranged.

Unless otherwise specified all the following changes were upstreamed to the mainstream GDB repository. According to the GDB schedule, the GDB10 branch point is planned on 2020-05-15 with release on 2020-06-05. It's a challenge to see how much the GDB support can be improved by then for NetBSD![Read More] [0 comments]

 

Extending support for the NetBSD-7 branch

Typically, some time after releasing a new NetBSD major version (such as NetBSD 9.0), we will announce the end-of-life of the N-2 branch, in this case NetBSD-7.

We've decided to hold off on doing that to ensure our users don't feel rushed to perform a major version update on any remote machines, possibly needing to reach the machine if anything goes wrong.

Security fixes will still be made to the NetBSD-7 branch.

We hope you're all safe. Stay home.

[1 comment]