Improving the ptrace(2) API and preparing for LLVM-10.0


January 13, 2020 posted by Kamil Rytarowski

This month I have improved the NetBSD ptrace(2) API, removing one legacy interface with a few flaws and replacing it with two new calls with new features, and removing technical debt.

As LLVM 10.0 is branching now soon (Jan 15th 2020), I worked on proper support of the LLVM features for NetBSD 9.0 (today RC1) and NetBSD HEAD (future 10.0).[Read More] [0 comments]

 

Working towards LLDB on i386


January 13, 2020 posted by Michał Górny

Upstream describes LLDB as a next generation, high-performance debugger. It is built on top of LLVM/Clang toolchain, and features great integration with it. At the moment, it primarily supports debugging C, C++ and ObjC code, and there is interest in extending it to more languages.

In February 2019, I have started working on LLDB, as contracted by the NetBSD Foundation. So far I've been working on reenabling continuous integration, squashing bugs, improving NetBSD core file support, extending NetBSD's ptrace interface to cover more register types and fix compat32 issues, fixing watchpoint and threading support.

Throughout December I've continued working on our build bot maintenance, in particular enabling compiler-rt tests. I've revived and finished my old patch for extended register state (XState) in core dumps. I've started working on bringing proper i386 support to LLDB.

[Read More] [0 comments]

 

GSoC 2019 Final Report: Incorporating the memory-hard Argon2 hashing scheme into NetBSD


January 12, 2020 posted by Jason High

Introduction

We successfully incorporated the Argon2 reference implementation into NetBSD/amd64 for our 2019 Google Summer of Coding project. We introduced our project here and provided some hints on how to select parameters here. For our final report, we will provide an overview of what changes were made to complete the project.

Incorporating the Argon2 Reference Implementation

The Argon2 reference implementation, available here, is available under both the Creative Commons CC0 1.0 and the Apache Public License 2.0. To import the reference implementation into src/external, we chose to use the Apache 2.0 license for this project.

During our initial phase 1, we focused on building the libargon2 library and integrating the functionality into the existing password management framework via libcrypt. Toward this end, we imported the reference implementation and created the "glue" to incorporate the changes into /usr/src/external/apache. The reference implementation is found in

m2$ ls /usr/src/external/apache2/argon2                                                                                    
Makefile dist     lib      usr.bin
The Argon2 reference implementation provides both a library and a binary. We build the libargon2 library to support libcrypt integration, and the argon2(1) binary to provide a userland command-line tool for evaluation. To build the code, we add MKARGON2 to bsd.own.mk
_MKVARS.yes= \
	...
        MKARGON2 \
	...
and add the following conditional build to /usr/src/external/apache2/Makefile
.if (defined(MKARGON2) && ${MKARGON2} != "no")
SUBDIR+= argon2
.endif
After successfully building and installation, we have the following new files and symlinks
/usr/bin/argon2
/usr/lib/libargon2.a
/usr/lib/libargon2.so
/usr/lib/libargon2.so.1
/usr/lib/libargon2.so.1.0
To incorporate Argon2 into the password management framework of NetBSD, we focused on libcrypt. In /usr/src/lib/libcrypt/Makefile, we first check for MKARGON2
.if (defined(MKARGON2) && ${MKARGON2} != "no")
HAVE_ARGON2=1
.endif
If HAVE_ARGON2 is defined and enabled, we append the following to the build flags
.if defined(HAVE_ARGON2)
SRCS+=          crypt-argon2.c
CFLAGS+=        -DHAVE_ARGON2 -I../../external/apache2/argon2/dist/phc-winner
-argon2/include/
LDADD+=         -largon2 
.endif
As hinted above, our most significant addition to libcrypt is the file crypt-argon2.c. This file pulls in the functionality of libargon2 into libcrypt. Changes were also made to pw_gensalt.c to allow for parameter parsing and salt generation.

Having completed the backend support, we pull Argon2 into userland tools, such as pwhash(1), in the same way as above

.if ( defined(MKARGON2) && ${MKARGON2} != "no" )
CPPFLAGS+=      -DHAVE_ARGON2
.endif
Once built, we can specify Argon2 using the '-A' command-line argument to pwhash(1), followed by the Argon2 variant name, and any of the parameterized values specified in argon2(1). See our first blog post for more details. As an example, to generate an argon2id encoding of the password password using default parameters, we can use the following
m2# pwhash -A argon2id password
$argon2id$v=19$m=4096,t=3,p=1$.SJJCiU575MDnA8s$+pjT4JsF2eLNQuLPEyhRA5LCFG
QWAKsksIPl5ewTWNY
To simplify Argon2 password management, we can utilize passwd.conf(5) to apply Argon2 to a specified user or all users. The same parameters are accepted as for argon2(1). For example, to specify argon2i with non-default parameters for user 'testuser', you can use the following in your passwd.conf
m1# grep -A1 testuser /etc/passwd.conf 
testuser:
        localcipher = argon2i,t=6,m=4096,p=1
With the above configuration in place, we are able to support standard password management. For example
m1# passwd testuser
Changing password for testuser.
New Password:
Retype New Password:

m1# grep testuser /etc/master.passwd  
testuser:$argon2i$v=19$m=4096,t=6,p=1$PDd65qr6JU0Pfnpr$8YOMYcwINuKHoxIV8Q0FJHG+
RP82xtmAuGep26brilU:1001:100::0:0::/home/testuser:/sbin/nologin

Testing

The argon2(1) binary allows us to easily validate parameters and encoding. This is most useful during performance testing, see here. With argon2(1), we can specify our parameterized values and evaluate both the resulting encoding and timing.
m2# echo -n password|argon2 somesalt -id -p 3 -m 8
Type:           Argon2id
Iterations:     3
Memory:         256 KiB
Parallelism:    3
Hash:           97f773f68715d27272490d3d2e74a2a9b06a5bca759b71eab7c02be8a453bfb9
Encoded:        $argon2id$v=19$m=256,t=3,p=3$c29tZXNhbHQ$l/dz9ocV0nJySQ09LnSiqb
BqW8p1m3Hqt8Ar6KRTv7k
0.000 seconds
Verification ok
We provide one approach to evaluating Argon2 parameter tuning in our second post. In addition to manual testing, we also provide some ATF tests for pwhash, for both hashing and verification. These tests are focus on encoding correctness, matching known encodings to test results during execution.
/usr/src/tests/usr.bin/argon2

tp: t_argon2_v10_hash
tp: t_argon2_v10_verify
tp: t_argon2_v13_hash
tp: t_argon2_v13_verify


cd /usr/src/tests/usr.bin/argon2
atf-run

info: atf.version, Automated Testing Framework 0.20 (atf-0.20)
info: tests.root, /usr/src/tests/usr.bin/argon2

..

tc-so:Executing command [ /bin/sh -c echo -n password | \
argon2 somesalt -v 13 -t 2 -m 8 -p 1 -r ]
tc-end: 1567497383.571791, argon2_v13_t2_m8_p1, passed

...

Conclusion

We have successfully integrated Argon2 into NetBSD using the native build framework. We have extended existing functionality to support local password management using Argon2 encoding. We are able to tune Argon2 so that we can achieve reasonable performance on NetBSD. In this final post, we summarize the work done to incorporate the reference implementation into NetBSD and how to use it. We hope you can use the work completed during this project. Thank you for the opportunity to participate in the Google Summer of Code 2019 and the NetBSD project!

[0 comments]

 

Clang build bot now uses two-stage builds, and other LLVM/LLDB news


December 12, 2019 posted by Michał Górny

Upstream describes LLDB as a next generation, high-performance debugger. It is built on top of LLVM/Clang toolchain, and features great integration with it. At the moment, it primarily supports debugging C, C++ and ObjC code, and there is interest in extending it to more languages.

In February, I have started working on LLDB, as contracted by the NetBSD Foundation. So far I've been working on reenabling continuous integration, squashing bugs, improving NetBSD core file support, extending NetBSD's ptrace interface to cover more register types and fix compat32 issues, and fixing watchpoint support. In October 2019, I've finished my work on threading support (pending pushes) and fought issues related to upgrade to NetBSD 9.

November was focused on finally pushing the aforementioned patches and major buildbot changes. Notably, I was working on extending the test runs to compiler-rt which required revisiting past driver issues, as well as resolving new ones. More details on this below.

[Read More] [1 comment]

 

First release candidate for NetBSD 9.0 available!


December 02, 2019 posted by Martin Husemann

Four months after the start of the release engineering process for 9.0, the first (and hopefully only) release candidate is now available.

The NetBSD 9.0 release comes with many new features and lots of improvements over the NetBSD 8.1 release...

[Read More] [3 comments]

 

Debugging FFS Mount Failures


November 27, 2019 posted by Kamil Rytarowski

This report was written by Maciej Grochowski as a part of developing the AFL+KCOV project.

This report is a continuation of my previous work on Fuzzing Filesystems via AFL. You can find previous posts where I described the fuzzing (part1, part2) or my EuroBSDcon presentation.
In this part, we won't talk too much about fuzzing itself but I want to describe the process of finding root causes of File system issues and my recent work trying to improve this process.
This story begins with a mount issue that I found during my very first run of the AFL, and I presented it during my talk on EuroBSDcon in Lillehammer.

[Read More] [1 comment]

 

Board of Directors and Officers elected


November 20, 2019 posted by William J. Coldwell

Per the membership voting, we have seated the new Board of Directors of the NetBSD Foundation:

  • Taylor R. Campbell <riastadh@>
  • William J. Coldwell <billc@>
  • Michael van Elst <mlelstv@>
  • Thomas Klausner <wiz@>
  • Cherry G. Mathew <cherry@>
  • Pierre Pronchery <khorben@>
  • Leonardo Taccari <leot@>

We would like to thank Makoto Fujiwara <mef@> and Jeremy C. Reed <reed@> for their service on the Board of Directors during their term(s).

The new Board of Directors have voted in the executive officers for The NetBSD Foundation:

President:William J. Coldwell
Vice President: Pierre Pronchery
Secretary: Christos Zoulas
Assistant Secretary: Thomas Klausner
Treasurer: Christos Zoulas
Assistant Treasurer: Taylor R. Campbell

Thanks to everyone that voted and we look forward to a great 2020.

[2 comments]

 

LLDB Threading support now ready for mainline


November 09, 2019 posted by Michał Górny

Upstream describes LLDB as a next generation, high-performance debugger. It is built on top of LLVM/Clang toolchain, and features great integration with it. At the moment, it primarily supports debugging C, C++ and ObjC code, and there is interest in extending it to more languages.

In February, I have started working on LLDB, as contracted by the NetBSD Foundation. So far I've been working on reenabling continuous integration, squashing bugs, improving NetBSD core file support, extending NetBSD's ptrace interface to cover more register types and fix compat32 issues and fixing watchpoint support. Then, I've started working on improving thread support which is taking longer than expected. You can read more about that in my September 2019 report.

So far the number of issues uncovered while enabling proper threading support has stopped me from merging the work-in-progress patches. However, I've finally reached the point where I believe that the current work can be merged and the remaining problems can be resolved afterwards. More on that and other LLVM-related events happening during the last month in this report.

[Read More] [0 comments]

 

Stabilization of the ptrace(2) threads continued


November 04, 2019 posted by Kamil Rytarowski

I have introduced changes to make debuggers more reliable in threaded scenarios. Additionally I have revamped micro-UBSan runtime for newer Clang (version 10git). I have received the OK from core@ to switch our iconv(3) to POSIX conformant iconv(3) and I have adapted where possible and readily known in pkgsrc to the newer API. This month I continued to find a solution to the impasse in LLD that blocks adding NetBSD support.[Read More] [0 comments]

 

Stabilization of the ptrace(2) threads


October 10, 2019 posted by Kamil Rytarowski

I have introduced changes that make debuggers more reliable in threaded scenarios. Additionally, I have enhanced Leak Sanitizer support and introduced various improvements in the basesystem.[Read More] [1 comment]

 

Threading support in LLDB continued


October 05, 2019 posted by Michał Górny

Upstream describes LLDB as a next generation, high-performance debugger. It is built on top of LLVM/Clang toolchain, and features great integration with it. At the moment, it primarily supports debugging C, C++ and ObjC code, and there is interest in extending it to more languages.

In February, I have started working on LLDB, as contracted by the NetBSD Foundation. So far I've been working on reenabling continuous integration, squashing bugs, improving NetBSD core file support, extending NetBSD's ptrace interface to cover more register types and fix compat32 issues and fixing watchpoint support. Then, I've started working on improving thread support. You can read more about that in my July 2019 report.

I've been on vacation in August, and in September I've resumed the work on LLDB. I've started by fixing new regressions in LLVM suite, then improved my previous patches and continued debugging test failures and timeouts resulting from my patches.

[Read More] [0 comments]

 

EuroBSDCon 2019


September 25, 2019 posted by Kamil Rytarowski

Submitted by Maciej Grochowski.

This year This year EuroBSDCon took place in Lillehammer Norway. I had the pleasure to attend as a speaker with my talk about fuzzing the NetBSD filesystems.[Read More] [1 comment]