NetBSD Bloghttps://blog.netbsd.org/tnf/feed/entries/atom2024-03-17T12:20:22+00:00Apache Roller (incubating)https://blog.netbsd.org/tnf/entry/gsoc_reports_make_system_31GSoC Reports: Make system(3), popen(3) and popenve(3) use posix_spawn(3) internally (Final report)Nikita Ronja Gillmann2021-03-30T11:11:15+00:002022-02-24T10:36:27+00:00<i>This report was prepared by Nikita Ronja Gillmann as a part of Google Summer of Code 2020</i>
<p>This is my second and final report for the Google Summer of Code project I am working on for NetBSD.</p>
<p>
My code can be found at <a href="https://github.com/teknokatze/src/">github.com/teknokatze/src</a> in the gsoc2020 branch, at the time of writing some of it is still missing. The test facilities and logs can be found in <a href="https://github.com/teknokatze/gsoc2020/">github.com/teknokatze/gsoc2020</a>. A diff can be found at <a href="https://github.com/NetBSD/src/compare/trunk...teknokatze:gsoc2020">github</a> which will later be split into several patches before it is sent to QA for merging.
</p>
<p>
The initial and defined goal of this project was to <a href="https://blog.netbsd.org/tnf/entry/gsoc_reports_make_system_3">make system(3) and popen(3) use posix_spawn(3) internally</a>, which had been completed in June.
For the second part I was given the task to replace fork+exec calls in our standard shell (sh) in one scenario. Similar to the previous goal we determine through implementation if the initial motivation, to get performance improvements, is correct otherwise we collect metrics for why posix_spawn() in this case should be avoided. This second part meant in practice that I had to add and change code in the kernel, add a new public libc function, and understand shell internals.
</p>
<i>This report was prepared by Nikita Ronja Gillmann as a part of Google Summer of Code 2020</i>
<p>This is my second and final report for the Google Summer of Code project I am working on for NetBSD.</p>
<p>
My code can be found at <a href="https://github.com/nikicoon/src/">github.com//src</a> in the <i>gsoc2020</i> branch, at the time of writing some of it is still missing.
The test facilities and logs can be found in <a href="https://github.com/nikicoon/gsoc2020/">github.com/nikicoon/gsoc2020</a>.
A diff can be found at <a href="https://github.com/NetBSD/src/compare/trunk...nikicoon:gsoc2020">github</a> which will later be split into several patches before it is sent to QA for merging.
</p>
<p>
The initial and defined goal of this project was to <a href="https://blog.netbsd.org/tnf/entry/gsoc_reports_make_system_3">make system(3) and popen(3) use posix_spawn(3) internally</a>, which had been completed in June.
For the second part I was given the task to replace fork+exec calls in our standard shell (sh) in one scenario. Similar to the previous goal we determined through implementation if the initial motivation, to get performance improvements, is correct otherwise we collect metrics for why posix_spawn() in this case should be avoided.
This second part meant in practice that I had to add and change code in the kernel, add a new public libc function, and understand shell internals.
</p>
<h2>Summary of part 1</h2>
<p>
Prior work: In GSoC 2012 Charles Zhang <a href="https://blog.netbsd.org/tnf/entry/posix_spawn_syscall_added">added the posix_spawn syscall</a> which according to <a href="https://web.archive.org/web/20150905210045/http://netbsd-soc.sourceforge.net/projects/posix_spawn/">its SF repository</a> at the time (maybe even now, I have not looked very much into comparing all other systems and libcs + kernels) is an in-kernel implementation of posix_spawn which provides performance benefits compared to FreeBSD and other systems which had a userspace implementation (in 2012).
</p>
<p>
After 1 week of reading POSIX and writing code, 2 weeks of coding and another 1.5 weeks of bugfixes I have successfully implemented posix_spawn in usage in system(3) and popen(3) internally.</p><p>The biggest challenge for me was to understand POSIX, to read the standard. I am used to reading more formal books, but I can't remember working with POSIX Standard directly before.
</p>
<h3>system(3)</h3>
<p>system(3) was changed to use posix_spawnattr_ (where we used sigaction before) and posix_spawn (which replaced execve + vfork calls).</p>
<h3>popen(3) and popenve(3)</h3>
<p>
Since the popen and popenve implementation in NetBSD's libc use a couple of shared helper functions, I was able to change both functions while keeping the majority of the changes focused on (some of ) the helper functions (pdes_child).
</p>
<p>
pdes_child, an internal function in popen.c, now takes one more argument (<i>const char *cmd</i>) for the command to pass to posix_spawn which is called in pdes_child.
</p>
<p>
On a high level what happens in pdes_child() and popen is that we first lock the pidlist_mutex. Then we create a file file action list for all concurrent popen() / popenve() instances and the side of the pipe not necessary, and the move to stdin/stdout. We unlock the pidlist_mutex. Finally we return the list and destroy.
</p>
<p>
In the new version of this helper function which now handles the majority of what popen/popenve did, we have to initialize a file_actions object which by default contains no file actions for posix_spawn() to perform. Since we have to have error handling and a common return value for the functions calling pdes_child() and deconstruction, we make use of <i>goto</i> in some parts of this function.
</p>
<p>
The close() and dup2() actions now get replaced by corresponding file_actions syscalls, they are used to specify a series of actions to be performed by a posix_spawn operation.</p>
<p>After this series of actions, we call _readlockenv(), and call posix_spawn with the file_action object and the other arguments to be executed. If it succeeds, we return the pid of the child to popen, otherwise we return -1, in both cases we destroy the file_action object before we proceed.</p>
<p>
In popen and popenve our code has been reduced to the <i>pid == -1</i> branch, everything else happens in pdes_child() now.
</p>
<p>
After readlockenv we call pdes_child and pass it the command to execute in the posix_spawn'd child process; if pdes_child returns -1 we run the old error handling code. Likewise for popenve.</p>
<p>
The outcome of the first part is, that thanks to how we implement posix_spawn in NetBSD we reduced the syscalls being made for popen and system.
A full test with proper timing should indicate this, my reading was based on comparing old and new logs with ktrace and kdump.
</p>
<h2>sh, posix_spawn actions, libc and kernel - Part 2</h2>
<h3>Motivation</h3>
<p>
The main goal of part 2 of this project was to change sh(1) to
determine which simple cases of (v)fork + exec I could replace, and to
replace them with posix_spawn where it makes sense.</p>
<p>
fork needs to create a new address space by cloning the address space,
or in the case of vfork update at least some reference counts.
posix_spawn can avoid most of this as it creates the new address space from scratch.
</p>
<h3>Issues</h3>
<p>
The current posix_spawn as defined in POSIX has no good way to do tcsetpgrp, and we found
that <a href="https://github.com/fish-shell/fish-shell/issues/360">fish just avoids posix_spawn for foreground processes</a>.
</p>
<h3>Implementation</h3>
<p>
Since, roughly speaking, modern BSDs handle "#!" execution in the kernel (probably since before the 1990s, systems which didn't handle this started to disappear most likely in the mid to late 90s), our main concern so far was in the evalcmd function the default cmd switch case ('NORMALCMD').</p><p>After adjusting the function to use posix_spawn, I hit an issue in the execution of the curses application htop where htop would run but input would not be accepted properly (keysequences pressed are visible).
In pre-posix_spawn sh, every subprocess that sh (v)forked runs forkchild() to set up the subprocess's environment.
With posix_spawn, we need to arrange posix_spawn actions to do the same thing.
</p>
<p>
The intermediate resolution was to switch FORK_FG processes to fork+exec again. For foreground processes with job control we're in an interactive shell, so the performance benefit is small enough in this case to be negligible. It's really only for shell scripts that it matters.</p><p>Next I implemented a posix_spawn file_action, with the prototype
</p>
<pre>
<code>int posix_spawn_file_actions_addtcsetpgrp(posix_spawn_file_actions_t *fa, int fildes)</code>
</pre>
<p>
The kernel part of this was implemented inline in sys/kern/kern_exec.c, in the function handle_posix_spawn_file_actions() for the new case 'FAE_TCSETPGRP'.</p><p>The new version of the code is still in testing and debugging phase and at the time of writing not included in my repository (it will be published after Google Summer of Code when I'm done moving).
</p>
<h3>Future steps</h3>
<h4>posix_spawnp kernel implementation</h4>
<p>
According to a conversation with kre@, the posix_spawnp() implementation we have is just itterating over $PATH calling posix_spawn until it succeeds.
For some changes we might want a kernel implementation of posix_spawnp(), as the path search is supposed to happen in the kernel so the file actions are only ever run once:
</p>
<pre>
<code>
some of the file actions may be "execute once only",
they can't be repeated (eg: handling "set -C; cat foo >file" - file
can only be created once, that has to happen before the exec (as the fd
needs to be made stdout), and then the exec part of posix_spawn is
attempted - if that fails, when it can't find "cat" in $HOME/bin (or
whatever is first in $PATH) and we move along to the next entry (maybe /bin
doesn't really matter) then the repeated file action fails, as file now
exists, and "set -C" demands that we cannot open an already existing file
(noclobber mode). It would be nice for this if there were "clean up on
failure" actions, but that is likely to be very difficult to get right,
and each would need to be attached to a file action, so only those which
had been performed would result in cleanup attempts.
</code>
</pre>
<h4>Replacing all of fork+exec in sh</h4>
<p>
Ideally we could replace all of (v)fork + exec with posix_spawn.
According to my mentors there is pmap synchronisation as an impact of
constructing the vm space from scratch with (v)fork.
Less IPIs (inter-processor interrupts) matter for small processes too.
</p>
<h4>posix_spawn_file_action_ioctl</h4>
<p>
Future directions could involve a posix_spawn action for an arbitrary ioctl.
</p>
<h2>Thanks</h2>
<p>
My thanks go to fellow NetBSD developers for answering questions, most recently kre@ for sharing invaluable sh knowledge, Riastradh and Jörg as the mentors I've interacted with most of the time and for their often in-depth explanations as well as allowing me to ask questions I sometimes felt were too obvious. My friends, for sticking up with my "weird" working schedule. Lastly would like to thank the Google Summer of Code program for continuing through the ongoing pandemic and giving students the chance to work on projects full-time.
</p>https://blog.netbsd.org/tnf/entry/hitting_donation_milestone_financial_reportHitting donation milestone, financial report for 2020Maya Rashish2021-03-29T09:17:29+00:002021-03-29T10:32:22+00:00<p>
We nearly hit our donation milestone set after the release of 9.0 of $50,000.<br />
These donations have enabled us to fund significant paid work on NetBSD in 2020.
</p><p>
We nearly hit our 2020 donation milestone set after the release of 9.0 of $50,000.
These donations have enabled us to fund significant work on NetBSD in 2020 such as:
<ul>
<li>an aarch64 package build server, <a href="http://victory.netbsd.org/pkgsrc/packages/reports/HEAD/evbarm64-9.0/">victory.netbsd.org</a>. Thanks to Western Washington University for hosting this machine.</li>
<li><a href="https://blog.netbsd.org/tnf/entry/wifi_renewal_restarted">Modernizing wi-fi network stack</a> and release engineering work by Martin Husemann</li>
<li><a href="https://blog.netbsd.org/tnf/entry/lldb_work_concluded">LLDB support</a> by Michał Górny</li>
<li><a href="https://blog.netbsd.org/tnf/entry/accomplishment_of_porting_ptrace_2">ptrace and GDB improvements</a> by Kamil Rytarowski</li>
</ul>
</p>
<p>
If you are interested in seeing more work like this, please consider donating <a href="https://www.paypal.com/cgi-bin/webscr?cmd=_donations&business=paypal%40NetBSD.org¤cy_code=USD&source=url">via PayPal</a> or <a href="https://github.com/sponsors/NetBSD">GitHub sponsors</a>.
</p>
<p>
The <a href="https://www.netbsd.org/foundation/reports/financial/2020.html">financial report for 2020</a> is also available.
</p>
<p>
Note: We realize that this data is inconsistent with the website indicator of donations. This is due to the fact the website is updated manually in an error-prone process as the donations are processed. The financial report (just completed) is prepared separately using <a href="https://www.ledger-cli.org/">ledger</a>.
</p>https://blog.netbsd.org/tnf/entry/google_summer_of_code_20202Google Summer of Code 2020: [Final Report] Enhancing Syzkaller support for NetBSDKamil Rytarowski2020-10-19T13:20:28+00:002020-10-19T13:20:28+00:00This report was written by Ayushu Sharma as part of Google Summer of Code 2020.
<p>This post is a follow up of the <a href="https://blog.netbsd.org/tnf/entry/gsoc_reports_enhancing_syzkaller_support">first report</a> and <a href="https://blog.netbsd.org/tnf/entry/gsoc_reports_enhancing_syzkaller_support1">second report</a>. Post summarizes the work done during the third and final coding period for the Google Summer of Code (GSoc’20) project - <a href="https://wiki.netbsd.org/projects/project/syzkaller/">Enhance Syzkaller support for NetBSD</a></p>This report was written by Ayushu Sharma as part of Google Summer of Code 2020.
<p>This post is a follow up of the <a href="https://blog.netbsd.org/tnf/entry/gsoc_reports_enhancing_syzkaller_support">first report</a> and <a href="https://blog.netbsd.org/tnf/entry/gsoc_reports_enhancing_syzkaller_support1">second report</a>. Post summarizes the work done during the third and final coding period for the Google Summer of Code (GSoc’20) project - <a href="https://wiki.netbsd.org/projects/project/syzkaller/">Enhance Syzkaller support for NetBSD</a></p>
<h2>Sys2syz</h2>
<p>Sys2syz would give an extra edge to Syzkaller for NetBSD. It has a potential of efficiently automating the conversion of syscall definitions to syzkaller’s grammar. This can aid in increasing the number of syscalls covered by Syzkaller significantly with the minimum possibility of manual errors. Let’s delve into its internals.</p>
<h2>A peek into Syz2syz Internals</h2>
<p>This tool parses the source code of device drivers present in C to a format which is compatible with grammar customized for syzkaller. Here, we try to cull the details of the target device by compiling, and then collocate the details with our python code. For further details about proposed design for the tool, refer to <a href="https://blog.netbsd.org/tnf/entry/gsoc_reports_enhancing_syzkaller_support1">previous post</a>.<p>
<p>Python code follows 4 major steps:<p>
<ul>
<li><a href="https://github.com/ais2397/sys2syz/blob/master/core/Extractor.py"><b>Extractor.py</b></a> - Extraction of all ioctl commands of a given device driver along with their arguments from the header files.</li>
<li><a href="https://github.com/ais2397/sys2syz/blob/master/core/Bear.py"><b>Bear.py</b></a> - Preprocessing of the device driver's files using compile_commands.json generated during the setup of tool using Bear.</li>
<li><a href="https://github.com/ais2397/sys2syz/blob/master/core/C2xml.py"><b>C2xml.py</b></a> - XML files are generated by running c2xml on preprocessed device files. This eases the process of fetching the information related to arguments of commands</li>
<li><a href="https://github.com/ais2397/sys2syz/blob/master/core/Description.py"><b>Description.py</b></a> - Generates descriptions for the ioctl commands and their arguments (builtin-types, arrays, pointers, structures and unions) using the XML files.</li>
</ul>
<h3>Extraction:</h3>
<p>This step involves fetching the possible ioctl commands for the target device driver and getting the files which have to be included in our dev_target.txt file. We have already seen all the commands for device drivers are defined in a specific way. These commands defined in the header files need to be grepped along with the major details, regex comes in as a rescue for this</p>
<pre>
<code>
io = re.compile("#define\s+(.*)\s+_IO\((.*)\).*")
iow = re.compile("#define\s+(.*)\s+_IOW\((.*),\s+(.*),\s+(.*)\).*")
ior = re.compile("#define\s+(.*)\s+_IOR\((.*),\s+(.*),\s+(.*)\).*")
iowr = re.compile("#define\s+(.*)\s+_IOWR\((.*),\s+(.*),\s+(.*)\).*")
</code>
</pre>
<p>
Code scans through all the header files present in the target device folder and extracts all the commands along with their details using compiled regex expressions. Details include the direction of buffer(null, in, out, inout) based on the types of Ioctl calls(_IO, _IOR, _IOW, _IOWR) and the argument of the call. These are stored in a file named ioctl_commands.txt at location out/<target_name>.
Example output:
<pre>
<code>
out, I2C_IOCTL_EXEC, i2c_ioctl_exec_t
</code>
</pre>
<h3>Preprocessing:</h3>
<p>Preprocessing is required for getting XML files, about which we would look in the next step. Bear plays a major role when it comes to preprocessing C files. It records the commands executed for building the target device driver. This step is performed when setup.sh script is executed.</p>
<p>Extracted commands are modified with the help of parse_commands() function to include ‘-E’ and ‘-fdirectives’ flags and give it a new output location. Commands extracted by this function are then used by the compile_target function which filters out the unnecessary flags and generates preprocessed files in our output directory.</p>
<h3>Generating XML files</h3>
<p>Run C2xml on the preprocessed files to fetch XML files which stores source code in a tree-like structure, making it easier to collect all the information related to each and every element of structures, unions etc. For eg: </p>
<pre>
<code>
<symbol type="struct" id="_5970" file="am2315.i" start-line="13240" start-col="16" end-line="13244" end-col="11" bit-size="96" alignment="4" offset="0">
<symbol type="node" id="_5971" ident="ipending" file="am2315.i" start-line="13241" start-col="33" end-line="13241" end-col="41" bit-size="32" alignment="4" offset="0" base-type-builtin="unsigned int"/<
<symbol type="node" id="_5972" ident="ilevel" file="am2315.i" start-line="13242" start-col="33" end-line="13242" end-col="39" bit-size="32" alignment="4" offset="4" base-type-builtin="int"/>
<symbol type="node" id="_5973" ident="imasked" file="am2315.i" start-line="13243" start-col="33" end-line="13243" end-col="40" bit-size="32" alignment="4" offset="8" base-type-builtin="unsigned int"/>
</symbol>
<symbol type="pointer" id="_5976" file="am2315.i" start-line="13249" start-col="14" end-line="13249" end-col="25" bit-size="64" alignment="8" offset="0" base-type-builtin="void"/>
<symbol type="array" id="_5978" file="am2315.i" start-line="13250" start-col="33" end-line="13250" end-col="39" bit-size="288" alignment="4" offset="0" base-type-builtin="unsigned int" array-size="9"/>
</code>
</pre>
<p>We would further see how attributes like - idents, id, type, base-type-builtin etc conveniently helps us to analyze code and generate descriptions in a trouble-free manner .
</p>
<h3>Descriptions.py</h3>
<p>Final part, which offers a txt file storing all the required descriptions as its output. Here, information from the xml files and ioctl_commands.txt are combined together to generate descriptions of ioctl commands and their arguments.</p>
<p>Xml files for the given target device are parsed to form trees, </p>
<pre>
<code>
for file in (os.listdir(self.target)):
tree = ET.parse(self.target+file)
self.trees.append(tree)
</code>
</pre>
<p>We then traverse through these trees to search for the arguments of a particular ioctl command (particularly _IOR, _IOW, _IOWR commands) by the name of the argument. Once an element with the same value for ident attribute is found, attributes of the element are further examined to get its type. Possible types for these arguments are - struct, union, enum, function, array, pointer, macro and node. Using the type information we determine the way to define the element in accordance with syzkaller’s grammar syntax.
</p>
<p>Building structs and unions involves defining their elements too, XML makes it easier. Program analyses each and every element which is a child of the root (struct/union) and generates its definitions. A dictionary helps in tracking the structs/unions which have been already built. Later, the dictionary is used to pretty print all the structs and union in the output file. Here is a code snippet which depicts the approach</p>
<pre>
<code>
name = child.get("ident")
if name not in self.structs_and_unions.keys():
elements = {}
for element in child:
elem_type = self.get_type(element)
elem_ident = element.get("ident")
if elem_type == None:
elem_type = element.get("type")
elements[element.get("ident")] = elem_type
element_str = ""
for element in elements:
element_str += element + "\t" + elements[element] + "\n"
self.structs_and_unions[name] = " {\n" + element_str + "}\n"
return str(name)
</code>
</pre>
<p>Task of creating descriptions for arrays is made simpler due to the attribute - `array-size`.
When it comes to dealing with pointers, syzkaller needs the user to fill in the direction of the pointer. This has already been taken care of while analyzing the ioctl commands in Extractor.py. The second argument with in/out/inout as its possible value depends on ‘fun’ macros - _IOR, _IOW, _IOWR respectively. </p>
<p>There is another category named as nodes which can be distinguished using the base-type-builtin and base-type attributes.
</p>
<h2>Result</h2>
<p>Once the setup script for sys2syz is executed, sys2syz can be used for a certain target_device file by executing the python wrapper script (<a href="https://github.com/ais2397/sys2syz/blob/master/sys2syz.py">sys2syz.py</a>) with :</p>
<pre>
<code>#bin/sh
python sys2syz.py -t <absolute_path_to_device_driver_source> -c compile_commands.json -v
</code>
</pre>
<p>This would generate a dev_<device_driver>.txt file in the out directory. An example description file autogenerated by sys2syz for i2c device driver.</p>
<pre>
<code>
#Autogenerated by sys2syz
include <i2c_io.h>
resource fd_i2c[fd]
syz_open_dev$I2C(dev ptr[in, string["/dev/i2c"]], id intptr, flags flags[open_flags]) fd_i2c
ioctl$I2C_IOCTL_EXEC(fd fd_i2c, cmd const[I2C_IOCTL_EXEC], arg ptr[out, i2c_ioctl_exec])
i2c_ioctl_exec {
iie_op flags[i2c_op_t_flags]
iie_addr int16
iie_buflen len[iie_buf, intptr]
iie_buf buffer[out]
iie_cmdlen len[iie_cmd, intptr]
iie_cmd buffer[out]
}
</code>
</pre>
<h2>Future Work</h2>
<p>Though we have a basic working structure of this tool, yet a lot has to be worked upon for leveling it up to make the best of it. Perfect goals would be met when there would be least of manual labor needed. Sys2syz still looks forward to automating the detection of macros used by the flag types in syzkaller. List of to-dos also includes extending syzkaller’s support for generation of description of syscalls.</p>
<p>Some other yet-to-be-done tasks include-
<ul>
<li> Generating descriptions for function type </li>
<li> Calculating attributes for structs and unions </li>
</ul>
<h2>Summary</h2>
<p>We have surely reached closer to our goals but the project needs active involvement and incremental updates to scale it up to its full potential. Looking forward to much more learning and making more contribution to NetBSD community.</p>
<p>Atlast, a word of thanks to my mentors William Coldwell, Siddharth Muralee, Santhosh Raju and Kamil Rytarowski as well as the NetBSD organization for being extremely supportive. Also, I owe a big thanks to Google for giving me such a glaring opportunity to work on this project.</p>https://blog.netbsd.org/tnf/entry/the_gnu_gdb_debugger_and4The GNU GDB Debugger and NetBSD (Part 5) Kamil Rytarowski2020-10-07T17:16:53+00:002020-10-07T17:24:34+00:00The NetBSD developers maintain two copies of GDB:
<ul>
<li>One in the base-system that includes a significant set of local patches.</li>
<li>Another one in pkgsrc whose patching is limited to mostly build fixes.</li>
</ul>
<p>
The base-system version of GDB (GPLv3) still relies on local patching to work.
I have set a goal to reduce the number of custom patches to bare minimum, ideally achieving the state of GDB working without any local modifications at all.The NetBSD developers maintain two copies of GDB:
<ul>
<li>One in the base-system that includes a significant set of local patches.</li>
<li>Another one in pkgsrc whose patching is limited to mostly build fixes.</li>
</ul>
<p>
The base-system version of GDB (GPLv3) still relies on local patching to work.
I have set a goal to reduce the number of custom patches to bare minimum, ideally achieving the state of GDB working without any local modifications at all.
<p>
<h2>GDB changes</h2>
<p>
Last month, the NetBSD/amd64 support was merged into gdbserver.
This month, the gdbserver target support was extended to
<a href="https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=8b667faedf6012048f1f6e71785b1ac1412b8a9c">NetBSD/i386</a> and
<a href="https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=8e1d09292902cff8325b08a64fa5a918c7f9aa4f">NetBSD/aarch64</a>.
The gdbserver and gdb code was cleaned up, refactored and made capable of introducing even more NetBSD targets.
<p>
Meanwhile, the NetBSD/i386 build of GDB was
<a href="https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=1eb6eb795fd3479c97d8aadc4f70d6afad5f8511">fixed</a>.
The missing include of x86-bsd-nat.h as a common header was added to i386-bsd-nat.h.
The i386 GDB code for BSD contained a runtime assert that verified whether the locally hardcoded
<var>struct sigcontext</var> is compatible with the system headers. In reality, the system headers are no longer using
this structure since 2003, after the switch to ucontext_t, and the validating code was no longer
effective. After the switch to newer GCC, this was reported as a unused local variable by the compiler.
I have decided to <a href="https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=6ff330351e7741774c4b7ac1214cf7d73c7eac4d">remove</a> the check on NetBSD entirely.
This was followed up by a small <a href="https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=064280be25f2ff27477ce649f01a70d42ddad2ae">build fix</a>.
<p>
The NetBSD team has noticed that the GDB's agent.cc code contains a portability bug and prepared a local fix.
The traditional behavior of the BSD kernel is that passing random values of sun_len (part of sockaddr_un)
can cause failures. In order to prevent the problems, the sockaddr_un structure is now zeroed before use.
I've reimplemented the fix and successfully
<a href="https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=e2a2a24a8e78427ff8667d625f5befbe88c328bb">upstreamed it</a>.
<p>
In order to easily resolve the issue with environment hardening enforced by PaX MPROTECT,
I've <a href="https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=91e5e8db334b9a87c54f03982dfa0c88e3c9d7a1">introduced
a runtime warning whenever byte transfers betweeen the debugee and debugger occur with the EACCES errno code</a>.
<p>
<h2>binutils changes</h2>
<p>
I've added support for NetBSD/aarch64 upstream, in
<a href="https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=c0b313441717b65569edb01bf9984d2066d899de">GNU BFD</a> and
<a href="https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=cc8b27f89cc8d0fde4532134c19c40c47a023abd">GNU GAS</a>.
NetBSD still carries local patches for the GNU binutils components, and GNU ld does not build out of the box on NetBSD/aarch64.
<p>
<h2>Summary</h2>
<p>
The NetBSD support in GNU binutils and GDB is improving promptly, and the most popular
platforms of amd64, i386 and aarch64 are getting proper support out of the box, without downstream patches.
The remaining patches for these CPUs include: streamlining kgdb support,
adding native GDB support for aarch64,
upstreaming local modifications from the GNU binutils components (especially BFD and ld) and introducing portability enhancements
in the dependent projects like libiberty and gnulib.
Then, the remaining work is to streamline support for the remaining CPUs (Alpha, VAX, MIPS, HPPA, IA64, SH3, PPC, etc.),
to develop the missing generic features (such as listing open file descriptors for the specified process) and
to fix failures in the regression test-suite.https://blog.netbsd.org/tnf/entry/google_summer_of_code_20201Google Summer of Code 2020: [Final Report] RumpKernel Syscall FuzzingKamil Rytarowski2020-09-25T21:53:00+00:002020-10-19T13:22:39+00:00This report was prepared by Aditya Vardhan Padala as a part of Google Summer of Code 2020
<p>This post is the third update to the project RumpKernel Syscall Fuzzing.</p>
<p>Part1 - <a href="https://blog.netbsd.org/tnf/entry/gsoc_reports_fuzzing_rumpkernel_syscalls1">https://blog.netbsd.org/tnf/entry/gsoc_reports_fuzzing_rumpkernel_syscalls1</a></p>
<p>Part2 - <a href="https://blog.netbsd.org/tnf/entry/gsoc_reports_fuzzing_rumpkernel_syscalls">https://blog.netbsd.org/tnf/entry/gsoc_reports_fuzzing_rumpkernel_syscalls</a></p>
<p>The first and second coding period was entirely dedicated to fuzzing rumpkernel syscalls using hongfuzz. Initially a dumb fuzzer was developed to start fuzzing but it soon reached its limits.</p>
<p>For the duration of second coding peroid we concentrated on crash reproduction and adding grammar to the fuzzer which yielded in better results as we tested on a bug in ioctl with grammar. Although this works for now crash reproduction needs to be improved to generate a working c reproducer.</p>
<p>For the last coding period I have looked into the internals of syzkaller to understand how it pregenerates input and how it mutates data. I have continued to work on integrating <a href="https://github.com/rumpkernel/buildrump.sh">buildrump.sh</a> with build.sh. buildrump eases the task fo building the rumpkernel on any host for any target.</p>
<p><a href="https://github.com/rumpkernel/buildrump.sh">buildrump.sh</a> is like a wrapper around build.sh to build the tools and rumpkernel from the source relevant to rumpkernel. So I worked to get buildrump.sh working with netbsd-src. Building the toolchain was successfull from netbsd-src. So binaries like rumpmake work just fine to continue building the rumpkernel.</p>
<p>But the rumpkernel failed to build due to some warnings and errors similar to the following. It can be due to the fact that buildrump.sh has been dormant recently I faced a lot of build issues.</p>
<pre><code><span class="hljs-attribute">nbmake[2]</span>: nbmake[2]: don't know how to make /root/buildrump.sh/obj/dest.stage/usr/lib/crti.o. Stop
<span class="crystal">nbmake[<span class="hljs-number">2</span>]: stopped in /root/buildrump.sh/src/<span class="hljs-class"><span class="hljs-keyword">lib</span>/<span class="hljs-title">librumpuser</span></span>
>> <span class="hljs-symbol">ERROR:</span>
>> make /root/buildrump.sh/obj/Makefile.first dependall</span>
</code></pre><p>Few of the similar errors were easily fixed but I couldn't integrate it during the time span of the coding period.</p>
<p><b>To Do</b></p>
<ul>
<li>Research more on grammar definition and look into the existing grammar fuzzers for a better understanding of generating grammar.</li>
<li>Integrate <a href="https://github.com/ais2397/sys2syz">syz2sys</a> with the existing fuzzer to include grammar generation for better results.</li>
</ul>
<p>GSoC with NetBSD has been an amazing journey throughout, in which I had a chance to learn from awesome people and work on amazing projects. I will continue to work on the project to achieve the goal of integrating my fuzzer with OSS Fuzz. I thank my mentors Siddharth Muralee, Maciej Grochowski, Christos Zoulas for their support and Kamil for his continuous guidance.</p>https://blog.netbsd.org/tnf/entry/google_summer_of_code_2020Google Summer of Code 2020: [Final Report] Curses Library Automated TestingKamil Rytarowski2020-09-25T21:50:01+00:002020-10-19T13:24:55+00:00This report was prepared by Naman Jain as a part of Google Summer of Code 2020
<p>
My GSoC project under NetBSD involves the development of the test framework of curses. This is the final blog report in a series of blog reports; you can look at the <a href="https://blog.netbsd.org/tnf/entry/gsoc_reports_curses_library_automated" rel="nofollow">first report</a> and <a href="https://blog.netbsd.org/tnf/entry/gsoc_2020_second_evaluation_report" rel="nofollow">second report</a> of the series.
<p>The first report gives a brief introduction of the project and some insights into the curses testframe through its architecture and language. To someone who wants to contribute to the test suite, this blog can act as the quick guide of how things work internally. Meanwhile, the second report discusses some of the concepts that were quite challenging for me to understand. I wanted to share them with those who may face such a challenge. Both of these reports also cover the progress made in various phases of the Summer of Code.</p>This report was prepared by Naman Jain as a part of Google Summer of Code 2020
<p>
My GSoC project under NetBSD involves the development of the test framework of curses. This is the final blog report in a series of blog reports; you can look at the <a href="https://blog.netbsd.org/tnf/entry/gsoc_reports_curses_library_automated" rel="nofollow">first report</a> and <a href="https://blog.netbsd.org/tnf/entry/gsoc_2020_second_evaluation_report" rel="nofollow">second report</a> of the series.
<p>The first report gives a brief introduction of the project and some insights into the curses testframe through its architecture and language. To someone who wants to contribute to the test suite, this blog can act as the quick guide of how things work internally. Meanwhile, the second report discusses some of the concepts that were quite challenging for me to understand. I wanted to share them with those who may face such a challenge. Both of these reports also cover the progress made in various phases of the Summer of Code.</p>
<p>This being the final report in the series, I would love to share my experience throughout the project. I would be sharing some of the learning as well as caveats that I faced in the project.</p>
<h2><a id="user-content-challenges-and-caveats" class="anchor" aria-hidden="true" href="#challenges-and-caveats"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Challenges and Caveats</h2>
<p>By the time my application for GSoC was submitted, I had gained some knowledge about the curses library and the testing framework. Combined with compiler design and library testing experience, that knowledge proved useful but not sufficient as I progressed through the project. There were times when, while writing a test case, you have to look into documentation from various sources, be it NetBSD, FreeBSD, Linux, Solaris, etc. One may find questioning his understanding of the framework, documentation, or even curses itself. This leads to the conclusion that for being a tester, one has to become a user first. That made me write minimal programs to understand the behavior. The experience was excellent, and I felt amazed by the capability and complexity of curses.</p>
<h2><a id="user-content-learnings" class="anchor" aria-hidden="true" href="#learnings"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Learnings</h2>
<p>The foremost learning is from the experience of interacting with the open-source community and feeling confident in my abilities to contribute. Understanding the workflows; following the best practices like considering the maintainability, readability, and simplicity of the code were significant learning.</p>
<p>The project-specific learning was not limited to test-framework but a deeper understanding of curses as I have to browse through codes for the functions tested. As <a href="https://www.linusakesson.net/programming/tty/" rel="nofollow">this</a> blog says, getting the TTY demystified was a long-time desire, which got fulfilled to some extent.</p>
<h2><a id="user-content-some-tests-from-test-suite" class="anchor" aria-hidden="true" href="#some-tests-from-test-suite"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Some tests from test suite</h2>
<p>In this section, I would discuss a couple of tests of the test suite written during the third phase of GSoC. Curses input model provides a variety of ways to obtain input from keyboard. We will consider 2 tests <code>keypad</code> and <code>halfdelay</code> that belong to input processing category but are somewhat unrelated.</p>
<h3><a id="user-content-keypad-processing" class="anchor" aria-hidden="true" href="#keypad-processing"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Keypad Processing</h3>
<p>An application can enable or disable the tarnslation of keypad using <code>keypad()</code> function. When translation is enabled, curses attempts to translate input sequence into a single key code. If disabled, curses passes the input as it is and any interpretation has to be made by application.</p>
<pre><code>include window
call $FALSE is_keypad $win1
input "\eOA"
call 0x1b wgetch $win1
call OK keypad $win1 $TRUE
input "\eOA"
call $KEY_UP wgetch $win1
# disable assembly of KEY_UP
call OK keyok $KEY_UP $FALSE
input "\eOA"
call 0x1b wgetch $win1
</code></pre>
<p>As keypad translation is disabled by default, on input of '\eOA', the input sequence is passed as it is and only '\e' (0x1b is hex code) is received on <code>wgetch()</code>. If we enable the translation, then the same input is translated as KEY_UP. In curses, one can disable assembly of specific key symbols using <code>keyok()</code>. See related <a href="http://man.netbsd.org/keypad.3" rel="nofollow">man page</a>.</p>
<h3><a id="user-content-input-mode" class="anchor" aria-hidden="true" href="#input-mode"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Input Mode</h3>
<p>Curses lets the application control the effect of input using four input modes; cooked, cbreak, half-delay, raw. They specify the effect of input in terms of echo-ing and delay. We will discuss about the <code>halfdelay</code> mode. The half-delay mode specifies how quickly certain curses function return to application when there is no terminal input waiting since the function is called.</p>
<pre><code>include start
delay 1000
# input delay 1000 equals to 10 tenths of seconds
# getch must fail for halfdelay(5) and pass for halfdelay(15)
input "a"
call OK halfdelay 15
call 0x61 getch
call OK halfdelay 5
input "a"
call -1 getch
</code></pre>
<p>We have set the delay for feeding input to terminal with delay of 1s(10 tenths of second). If the application sets the halfdelay to 15, and makes a call to <code>getch()</code> it receives the input. But it fails to get the input with haldelay set to 5. See related <a href="http://man.netbsd.org/halfdelay.3" rel="nofollow">man page</a>.</p>
<h2><a id="user-content-project-work" class="anchor" aria-hidden="true" href="#project-work"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Project Work</h2>
<p>The <a href="https://github.com/NamanJain8/curses">work</a> can be merged into organisation repository <a href="https://github.com/NetBSD/src">https://github.com/NetBSD/src</a> under <a href="https://github.com/NetBSD/src/tree/trunk/tests/lib/libcurses">tests/lib/libcurses</a>.</p>
<p>This project involved:</p>
<ol>
<li>Improvement in testframework:
<ul>
<li>Automation of the checkfile generation.</li>
<li>Enhnacement of support for complex character</li>
<li>Addition of small features and code refactoring</li>
</ul>
</li>
<li>Testing and bug reports:
<ul>
<li>Tests for a family of routines like wide character, complex character, line drawing, box drawing, pad, window operations, cursor manipulations, soft label keys, input-output stream, and the ones involving their interactions.</li>
<li>Raising a bunch of Problem Report (PR) under <code>lib</code> category some of which have been fixed. The list of PRs raised can be found <a href="https://github.com/NamanJain8/curses/blob/master/reports/problem-reports.md">here</a></li>
</ul>
</li>
</ol>
<h2><a id="user-content-future-work" class="anchor" aria-hidden="true" href="#future-work"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Future Work</h2>
<ul>
<li>The current testframe supports complex character, but the support needs to be extended for its string. This will enable testing of <code>[mv][w]add_wch[n]str</code>, <code>[mv][w]in_wchstr</code> family of routines.</li>
<li>Some of the tests for teminal manipulation routines like <code>intrflush</code>, <code>def_prog_mode</code>, <code>typeahead</code>, <code>raw</code>, etc. are not there in test suite.</li>
<li>Not specifically related to the framework, but the documentation for wide character as well as complex character routines need to be added.</li>
</ul>
<h2><a id="user-content-acknowledgements" class="anchor" aria-hidden="true" href="#acknowledgements"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Acknowledgements</h2>
<p>I want to extend my heartfelt gratitude to my mentor Mr. Brett Lymn, who helped me through all the technical difficulties and challenges I faced. I also thank my mentor Martin Huseman for valuable suggestions and guidance at various junctures of the project. A special thanks to Kamil Rytarowski for making my blogs published on the NetBSD site.</p>https://blog.netbsd.org/tnf/entry/the_gnu_gdb_debugger_and3The GNU GDB Debugger and NetBSD (Part 4)Kamil Rytarowski2020-09-10T21:13:01+00:002020-09-10T21:17:53+00:00The NetBSD team of developers maintains two copies of GDB:
<ul>
<li>One in the base-system with a stack of local patches.</li>
<li>One in pkgsrc with mostly build fix patches.</li>
</ul>
<p>
The base-system version of GDB (GPLv3) still relies on a set of local patches.
I set a goal to reduce the local patches to bare minimum, ideally reaching no local modifications at all.The NetBSD team of developers maintains two copies of GDB:
<ul>
<li>One in the base-system with a stack of local patches.</li>
<li>One in pkgsrc with mostly build fix patches.</li>
</ul>
<p>
The base-system version of GDB (GPLv3) still relies on a set of local patches.
I set a goal to reduce the local patches to bare minimum, ideally reaching no local modifications at all.
<p>
<h1>GDB changes</h1>
<p>
Over the past month I worked on gdbserver for NetBSD/amd64 and finally
<a href="https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=62ba50486f1146f0cfd33074fc127fe00a02e87e">upstreamed</a> it to the GDB mainline, just in time for GDB 10.
<p>
What is gdbserver? Let's quote the official
<a href="https://sourceware.org/gdb/onlinedocs/gdb/Server.html">GDB documentation</a>:
<p>
<i><quote>
gdbserver is a control program for Unix-like systems, which allows you to connect your program with a remote GDB via target remote or target extended-but without linking in the usual debugging stub.
<p>
gdbserver is not a complete replacement for the debugging stubs, because it requires essentially the same operating-system facilities that GDB itself does. In fact, a system that can run gdbserver to connect to a remote GDB could also run GDB locally! gdbserver is sometimes useful nevertheless, because it is a much smaller program than GDB itself. It is also easier to port than all of GDB, so you may be able to get started more quickly on a new system by using gdbserver. Finally, if you develop code for real-time systems, you may find that the tradeoffs involved in real-time operation make it more convenient to do as much development work as possible on another system, for example by cross-compiling. You can use gdbserver to make a similar choice for debugging.
<p>
GDB and gdbserver communicate via either a serial line or a TCP connection, using the standard GDB remote serial protocol. remote
</quote></i>
<p>
This illustrated that gdbserver is especially useful for debugging applications on embedded and thin devices, connected to
a controlling computer equipped with full distribution sources, toolchain, debugging information etc.
Eventually, this approach of gdb and gdbserver can replace the native gdb plugin entirely and spawn all connections
debugging sessions using this protocol.
This design decision was already introduced into LLDB, where remote process plugin is the only supported program
on Linux, NetBSD and highly recommended for other kernels.
<p>
I've picked amd64 as the first target as it's the easiest to develop and test.
<p>
An example debugging session looks like this:
<pre>
$ uname -rms
NetBSD 9.99.72 amd64
$ LC_ALL=C date
Thu Sep 10 22:43:10 CEST 2020
$ ./gdbserver/gdbserver --version
GNU gdbserver (GDB) 10.0.50.20200910-git
Copyright (C) 2020 Free Software Foundation, Inc.
gdbserver is free software, covered by the GNU General Public License.
This gdbserver was configured as "x86_64-unknown-netbsd9.99"
$ ./gdbserver/gdbserver localhost:1234 /usr/bin/nslookup
Process /usr/bin/nslookup created; pid = 26383
Listening on port 1234
</pre>
<p>
Then on the other terminal:
<p>
<pre>
$ ./gdb/gdb
GNU gdb (GDB) 10.0.50.20200910-git
Copyright (C) 2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-unknown-netbsd9.99".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
--Type <RET> for more, q to quit, c to continue without paging--
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word".
(gdb) target remote localhost:1234
Remote debugging using localhost:1234
Reading /usr/bin/nslookup from remote target...
warning: File transfers from remote targets can be slow. Use "set sysroot" to access files locally instead.
Reading /usr/bin/nslookup from remote target...
Reading symbols from target:/usr/bin/nslookup...
Reading /usr/bin/nslookup.debug from remote target...
Reading /usr/bin/.debug/nslookup.debug from remote target...
Reading /usr/libdata/debug//usr/bin/nslookup.debug from remote target...
Reading /usr/libdata/debug//usr/bin/nslookup.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/bin/nslookup.debug...
process 28353 is executing new program: /usr/bin/nslookup
Reading /usr/bin/nslookup from remote target...
Reading /usr/bin/nslookup from remote target...
Reading /usr/bin/nslookup.debug from remote target...
Reading /usr/bin/.debug/nslookup.debug from remote target...
Reading /usr/libdata/debug//usr/bin/nslookup.debug from remote target...
Reading /usr/libdata/debug//usr/bin/nslookup.debug from remote target...
Reading /usr/libexec/ld.elf_so from remote target...
Reading /usr/libexec/ld.elf_so from remote target...
Reading /usr/libexec/ld.elf_so.debug from remote target...
Reading /usr/libexec/.debug/ld.elf_so.debug from remote target...
Reading /usr/libdata/debug//usr/libexec/ld.elf_so.debug from remote target...
Reading /usr/libdata/debug//usr/libexec/ld.elf_so.debug from remote target...
warning: Invalid remote reply: timeout [kamil: repeated multiple times...]
Reading /usr/lib/libbind9.so.15 from remote target...
Reading /usr/lib/libisccfg.so.15 from remote target...
Reading /usr/lib/libdns.so.15 from remote target...
Reading /usr/lib/libns.so.15 from remote target...
Reading /usr/lib/libirs.so.15 from remote target...
Reading /usr/lib/libisccc.so.15 from remote target...
Reading /usr/lib/libisc.so.15 from remote target...
Reading /usr/lib/libkvm.so.6 from remote target...
Reading /usr/lib/libz.so.1 from remote target...
Reading /usr/lib/libblocklist.so.0 from remote target...
Reading /usr/lib/libpthread.so.1 from remote target...
Reading /usr/lib/libpthread.so.1.4.debug from remote target...
Reading /usr/lib/.debug/libpthread.so.1.4.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libpthread.so.1.4.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libpthread.so.1.4.debug from remote target...
Reading /usr/lib/libgssapi.so.11 from remote target...
Reading /usr/lib/libheimntlm.so.5 from remote target...
Reading /usr/lib/libkrb5.so.27 from remote target...
Reading /usr/lib/libcom_err.so.8 from remote target...
Reading /usr/lib/libhx509.so.6 from remote target...
Reading /usr/lib/libcrypto.so.14 from remote target...
Reading /usr/lib/libasn1.so.10 from remote target...
Reading /usr/lib/libwind.so.1 from remote target...
Reading /usr/lib/libheimbase.so.2 from remote target...
Reading /usr/lib/libroken.so.20 from remote target...
Reading /usr/lib/libsqlite3.so.1 from remote target...
Reading /usr/lib/libcrypt.so.1 from remote target...
Reading /usr/lib/libutil.so.7 from remote target...
Reading /usr/lib/libedit.so.3 from remote target...
Reading /usr/lib/libterminfo.so.2 from remote target...
Reading /usr/lib/libc.so.12 from remote target...
Reading /usr/lib/libgcc_s.so.1 from remote target...
Reading symbols from target:/usr/lib/libbind9.so.15...
Reading /usr/lib/libbind9.so.15.0.debug from remote target...
Reading /usr/lib/.debug/libbind9.so.15.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libbind9.so.15.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libbind9.so.15.0.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libbind9.so.15.0.debug...
Reading symbols from target:/usr/lib/libisccfg.so.15...
Reading /usr/lib/libisccfg.so.15.0.debug from remote target...
Reading /usr/lib/.debug/libisccfg.so.15.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libisccfg.so.15.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libisccfg.so.15.0.debug from remote target...
--Type <RET> for more, q to quit, c to continue without paging--
Reading symbols from target:/usr/libdata/debug//usr/lib/libisccfg.so.15.0.debug...
Reading symbols from target:/usr/lib/libdns.so.15...
Reading /usr/lib/libdns.so.15.0.debug from remote target...
Reading /usr/lib/.debug/libdns.so.15.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libdns.so.15.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libdns.so.15.0.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libdns.so.15.0.debug...
Reading symbols from target:/usr/lib/libns.so.15...
Reading /usr/lib/libns.so.15.0.debug from remote target...
Reading /usr/lib/.debug/libns.so.15.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libns.so.15.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libns.so.15.0.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libns.so.15.0.debug...
Reading symbols from target:/usr/lib/libirs.so.15...
Reading /usr/lib/libirs.so.15.0.debug from remote target...
Reading /usr/lib/.debug/libirs.so.15.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libirs.so.15.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libirs.so.15.0.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libirs.so.15.0.debug...
Reading symbols from target:/usr/lib/libisccc.so.15...
Reading /usr/lib/libisccc.so.15.0.debug from remote target...
Reading /usr/lib/.debug/libisccc.so.15.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libisccc.so.15.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libisccc.so.15.0.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libisccc.so.15.0.debug...
Reading symbols from target:/usr/lib/libisc.so.15...
Reading /usr/lib/libisc.so.15.0.debug from remote target...
Reading /usr/lib/.debug/libisc.so.15.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libisc.so.15.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libisc.so.15.0.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libisc.so.15.0.debug...
Reading symbols from target:/usr/lib/libkvm.so.6...
Reading /usr/lib/libkvm.so.6.0.debug from remote target...
Reading /usr/lib/.debug/libkvm.so.6.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libkvm.so.6.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libkvm.so.6.0.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libkvm.so.6.0.debug...
Reading symbols from target:/usr/lib/libz.so.1...
Reading /usr/lib/libz.so.1.0.debug from remote target...
Reading /usr/lib/.debug/libz.so.1.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libz.so.1.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libz.so.1.0.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libz.so.1.0.debug...
Reading symbols from target:/usr/lib/libblocklist.so.0...
Reading /usr/lib/libblocklist.so.0.0.debug from remote target...
Reading /usr/lib/.debug/libblocklist.so.0.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libblocklist.so.0.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libblocklist.so.0.0.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libblocklist.so.0.0.debug...
Reading symbols from target:/usr/lib/libgssapi.so.11...
Reading /usr/lib/libgssapi.so.11.0.debug from remote target...
Reading /usr/lib/.debug/libgssapi.so.11.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libgssapi.so.11.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libgssapi.so.11.0.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libgssapi.so.11.0.debug...
Reading symbols from target:/usr/lib/libheimntlm.so.5...
Reading /usr/lib/libheimntlm.so.5.0.debug from remote target...
Reading /usr/lib/.debug/libheimntlm.so.5.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libheimntlm.so.5.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libheimntlm.so.5.0.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libheimntlm.so.5.0.debug...
Reading symbols from target:/usr/lib/libkrb5.so.27...
Reading /usr/lib/libkrb5.so.27.0.debug from remote target...
Reading /usr/lib/.debug/libkrb5.so.27.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libkrb5.so.27.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libkrb5.so.27.0.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libkrb5.so.27.0.debug...
Reading symbols from target:/usr/lib/libcom_err.so.8...
Reading /usr/lib/libcom_err.so.8.0.debug from remote target...
Reading /usr/lib/.debug/libcom_err.so.8.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libcom_err.so.8.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libcom_err.so.8.0.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libcom_err.so.8.0.debug...
Reading symbols from target:/usr/lib/libhx509.so.6...
Reading /usr/lib/libhx509.so.6.0.debug from remote target...
Reading /usr/lib/.debug/libhx509.so.6.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libhx509.so.6.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libhx509.so.6.0.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libhx509.so.6.0.debug...
Reading symbols from target:/usr/lib/libcrypto.so.14...
Reading /usr/lib/libcrypto.so.14.0.debug from remote target...
Reading /usr/lib/.debug/libcrypto.so.14.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libcrypto.so.14.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libcrypto.so.14.0.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libcrypto.so.14.0.debug...
Reading symbols from target:/usr/lib/libasn1.so.10...
Reading /usr/lib/libasn1.so.10.0.debug from remote target...
Reading /usr/lib/.debug/libasn1.so.10.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libasn1.so.10.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libasn1.so.10.0.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libasn1.so.10.0.debug...
Reading symbols from target:/usr/lib/libwind.so.1...
Reading /usr/lib/libwind.so.1.0.debug from remote target...
Reading /usr/lib/.debug/libwind.so.1.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libwind.so.1.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libwind.so.1.0.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libwind.so.1.0.debug...
Reading symbols from target:/usr/lib/libheimbase.so.2...
Reading /usr/lib/libheimbase.so.2.0.debug from remote target...
Reading /usr/lib/.debug/libheimbase.so.2.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libheimbase.so.2.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libheimbase.so.2.0.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libheimbase.so.2.0.debug...
Reading symbols from target:/usr/lib/libroken.so.20...
Reading /usr/lib/libroken.so.20.0.debug from remote target...
Reading /usr/lib/.debug/libroken.so.20.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libroken.so.20.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libroken.so.20.0.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libroken.so.20.0.debug...
Reading symbols from target:/usr/lib/libsqlite3.so.1...
Reading /usr/lib/libsqlite3.so.1.4.debug from remote target...
Reading /usr/lib/.debug/libsqlite3.so.1.4.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libsqlite3.so.1.4.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libsqlite3.so.1.4.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libsqlite3.so.1.4.debug...
Reading symbols from target:/usr/lib/libcrypt.so.1...
Reading /usr/lib/libcrypt.so.1.0.debug from remote target...
Reading /usr/lib/.debug/libcrypt.so.1.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libcrypt.so.1.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libcrypt.so.1.0.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libcrypt.so.1.0.debug...
Reading symbols from target:/usr/lib/libutil.so.7...
Reading /usr/lib/libutil.so.7.24.debug from remote target...
Reading /usr/lib/.debug/libutil.so.7.24.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libutil.so.7.24.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libutil.so.7.24.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libutil.so.7.24.debug...
Reading symbols from target:/usr/lib/libedit.so.3...
Reading /usr/lib/libedit.so.3.1.debug from remote target...
Reading /usr/lib/.debug/libedit.so.3.1.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libedit.so.3.1.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libedit.so.3.1.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libedit.so.3.1.debug...
Reading symbols from target:/usr/lib/libterminfo.so.2...
Reading /usr/lib/libterminfo.so.2.0.debug from remote target...
Reading /usr/lib/.debug/libterminfo.so.2.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libterminfo.so.2.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libterminfo.so.2.0.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libterminfo.so.2.0.debug...
Reading symbols from target:/usr/lib/libc.so.12...
Reading /usr/lib/libc.so.12.217.debug from remote target...
Reading /usr/lib/.debug/libc.so.12.217.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libc.so.12.217.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libc.so.12.217.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libc.so.12.217.debug...
Reading symbols from target:/usr/lib/libgcc_s.so.1...
Reading /usr/lib/libgcc_s.so.1.0.debug from remote target...
Reading /usr/lib/.debug/libgcc_s.so.1.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libgcc_s.so.1.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libgcc_s.so.1.0.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libgcc_s.so.1.0.debug...
Reading /usr/libexec/ld.elf_so from remote target...
_rtld_debug_state () at /usr/src/libexec/ld.elf_so/rtld.c:1577
1577 __insn_barrier();
(gdb) b main
Breakpoint 1 at 0x211c00: file /usr/src/external/mpl/bind/bin/nslookup/../../dist/bin/dig/nslookup.c, line 990.
(gdb) c
Continuing.
Breakpoint 1, main (argc=1, argv=0x7f7fffffe768)
at /usr/src/external/mpl/bind/bin/nslookup/../../dist/bin/dig/nslookup.c:990
990 main(int argc, char **argv) {
(gdb) bt
#0 main (argc=1, argv=0x7f7fffffe768)
at /usr/src/external/mpl/bind/bin/nslookup/../../dist/bin/dig/nslookup.c:990
(gdb) info threads
Id Target Id Frame
* 1 Thread 28353.28353 main (argc=1, argv=0x7f7fffffe768)
at /usr/src/external/mpl/bind/bin/nslookup/../../dist/bin/dig/nslookup.c:990
(gdb) b pthread_setname_np
Breakpoint 2 at 0x7f7ff4e0c9e4: file /usr/src/lib/libpthread/pthread.c, line 792.
(gdb) c
Continuing.
[New Thread 28353.27773]
Thread 1 hit Breakpoint 2, pthread_setname_np (thread=0x7f7ff7e41000,
name=name@entry=0x7f7fffffe610 "work-0", arg=arg@entry=0x0)
at /usr/src/lib/libpthread/pthread.c:792
792 {
(gdb) info threads
Id Target Id Frame
* 1 Thread 28353.28353 pthread_setname_np (thread=0x7f7ff7e41000,
name=name@entry=0x7f7fffffe610 "work-0", arg=arg@entry=0x0)
at /usr/src/lib/libpthread/pthread.c:792
2 Thread 28353.27773 0x00007f7ff0aa623a in ___lwp_park60 () from target:/usr/lib/libc.so.12
(gdb) n
796 pthread__error(EINVAL, "Invalid thread",
(gdb) n
799 if (pthread__find(thread) != 0)
(gdb)
802 namelen = snprintf(newname, sizeof(newname), name, arg);
(gdb)
803 if (namelen >= PTHREAD_MAX_NAMELEN_NP)
(gdb)
806 cp = strdup(newname);
(gdb)
807 if (cp == NULL)
(gdb)
810 pthread_mutex_lock(&thread->pt_lock);
(gdb)
811 oldname = thread->pt_name;
(gdb)
812 thread->pt_name = cp;
(gdb)
813 (void)_lwp_setname(thread->pt_lid, cp);
(gdb)
814 pthread_mutex_unlock(&thread->pt_lock);
(gdb) n
816 if (oldname != NULL)
(gdb) n
isc_taskmgr_create (mctx=<optimized out>, workers=workers@entry=1, default_quantum=<optimized out>,
default_quantum@entry=0, nm=nm@entry=0x0, managerp=managerp@entry=0x418638 <taskmgr>)
at /usr/src/external/mpl/bind/lib/libisc/../../dist/lib/isc/task.c:1431
1431 for (i = 0; i < workers; i++) {
(gdb) info threads
Id Target Id Frame
* 1 Thread 28353.28353 isc_taskmgr_create (mctx=<optimized out>, workers=workers@entry=1,
default_quantum=<optimized out>, default_quantum@entry=0, nm=nm@entry=0x0,
managerp=managerp@entry=0x418638 <taskmgr>)
at /usr/src/external/mpl/bind/lib/libisc/../../dist/lib/isc/task.c:1431
2 Thread 28353.27773 "work-0" 0x00007f7ff0aa623a in ___lwp_park60 ()
from target:/usr/lib/libc.so.12
(gdb) dis 1
(gdb) b exit
Breakpoint 3 at 0x7f7ff0b530e0: exit. (2 locations)
(gdb) c
Continuing.
Thread 1 hit Breakpoint 2, pthread_setname_np (thread=0x7f7ff7e42c00,
name=name@entry=0x7f7ff5e6324e "isc-timer", arg=arg@entry=0x0)
at /usr/src/lib/libpthread/pthread.c:792
792 {
(gdb) dis 2
(gdb) c
Continuing.
Reading /usr/lib/i18n/libUTF8.so.5.0 from remote target...
Reading /usr/lib/i18n/libUTF8.so.5.0.debug from remote target...
Reading /usr/lib/i18n/.debug/libUTF8.so.5.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/i18n/libUTF8.so.5.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/i18n/libUTF8.so.5.0.debug from remote target...
</pre>
<p>
Then, back to the first terminal:
<pre>
> netbsd.org
Server: 62.179.1.62
Address: 62.179.1.62#53
Non-authoritative answer:
Name: netbsd.org
Address: 199.233.217.205
Name: netbsd.org
Address: 2001:470:a085:999::80
> exit
</pre>
<p>
<pre>
Thread 1 hit Breakpoint 3, exit (status=1) at /usr/src/lib/libc/stdlib/exit.c:55
55 {
(gdb) info threads
Id Target Id Frame
* 1 Thread 28353.28353 exit (status=1) at /usr/src/lib/libc/stdlib/exit.c:55
(gdb) bt
#0 exit (status=1) at /usr/src/lib/libc/stdlib/exit.c:55
#1 0x0000000000206122 in ___start ()
#2 0x00007f7ff7c0c840 in ?? () from target:/usr/libexec/ld.elf_so
#3 0x0000000000000001 in ?? ()
#4 0x00007f7fffffed20 in ?? ()
#5 0x0000000000000000 in ?? ()
(gdb) kill
Kill the program being debugged? (y or n) y
[Inferior 1 (process 28353) killed]
</pre>
<p>
<b>It worked!</b>
<p>
In order to get this functionality operational I had to implement multiple GDB functions, in particular: create_inferior,
post_create_inferior, attach, kill, detach, mourn, join, thread_alive,
resume, wait, fetch_registers, store_registers, read_memory, write_memory,
request_interrupt, supports_read_auxv, read_auxv,
supports_hardware_single_step, sw_breakpoint_from_kind,
supports_z_point_type, insert_point, remove_point,
stopped_by_sw_breakpoint, supports_qxfer_siginfo, qxfer_siginfo,
supports_stopped_by_sw_breakpoint, supports_non_stop,
supports_multi_process, supports_fork_events, supports_vfork_events,
supports_exec_events, supports_disable_randomization,
supports_qxfer_libraries_svr4, qxfer_libraries_svr4,
supports_pid_to_exec_file, pid_to_exec_file, thread_name,
supports_catch_syscall.
<p>
NetBSD is the first BSD and actually the first Open Source UNIX-like OS besides Linux to grow support for gdbserver.
<p>
<h1>Plan for the next milestone</h1>
<p>
Introduce AArch64 support for GDB/NetBSD.https://blog.netbsd.org/tnf/entry/gsoc_2020_report_2_fuzzingGSoC 2020: Report-2: Fuzzing the NetBSD Network Stack in a Rumpkernel EnvironmentKamil Rytarowski2020-08-30T13:21:57+00:002020-08-30T13:21:57+00:00This report was written by Nisarg S. Joshi as part of Google Summer of Code 2020.
<p>The objective of this project is to fuzz the various protocols and layers of the network stack of NetBSD using rumpkernel. This project is being carried out as a part of GSoC 2020. This blog post is regarding the project, the concepts and tools involved, the objectives and the current progress and next steps.</p>
<p>You can read the previous post/report <a href="http://blog.netbsd.org/tnf/entry/gsoc_reports_fuzzing_the_netbsd">here</a>.</p>This report was written by Nisarg S. Joshi as part of Google Summer of Code 2020.
<p>The objective of this project is to fuzz the various protocols and layers of the network stack of NetBSD using rumpkernel. This project is being carried out as a part of GSoC 2020. This blog post is regarding the project, the concepts and tools involved, the objectives and the current progress and next steps.</p>
<p>You can read the previous post/report <a href="http://blog.netbsd.org/tnf/entry/gsoc_reports_fuzzing_the_netbsd">here</a>.</p>
<p><strong>Overview of the work done:</strong></p>
<p>The major time of the phase 1 and 2 were spent in analyzing the input and output paths of the particular protocols being fuzzed. During that time, 5 major protocols of the internet stack were taken up:</p>
<ol>
<li>IPv4 (Phase 1)</li>
<li>UDP (Phase 1)</li>
<li>IPv6 (Phase 2)</li>
<li>ICMP (Phase 2)</li>
<li>Ethernet (Phase 2)</li>
</ol>
<p>Quite a good amount of time was spent in understanding the input and output processing functions of the particular protocols, the information gathered was to be applied in packet creation code for that protocol. This is important so that we know which parts of the packet can be kept random by the fuzzer based input and which part of the packet need to be set to proper fixed values. Fixing some values in the data packet to be correct is important so that the packet does not get rejected for trivial cases like IP Protocol Version or Internet Checksum. (The procedure to come up with the decisions and the code design and flow is explained in IPv4 Protocol section as an example)</p>
<p>For each protocol, mainly 2 things needed to be implemented: </p>
<ol>
<li>The Network Config: the topology for sending and receiving packets example using a TUN device or a TAP device, the socket used and so on. Configuring these devices was the first step in being able to send or receive packets</li>
<li>Packet Creation: Using the information gathered in the code walkthrough of the protocol functions, packet creation is decided where certain parts of the packet are kept fixed and others random from the fuzzer input itself. Doing so we try to gain maximum code coverage. Also one thing to be noted here, we should not randomly change the fuzzer input, rather do it deterministically following the same rules for each packet, otherwise the evolutionary fuzzer cannot easily grow the corpus.</li>
</ol>
<p>In the next section, a few of the protocols will be explained in detail.</p>
<p><strong>Protocols</strong></p>
<p>In this section we will talk about the various protocols implemented for fuzzing and talk about the approach taken to create a packet. </p>
<p><strong>IPv4:</strong></p>
<p>IPv4 stands for the Internet protocol version 4. It is one of the most widely used protocols in the internet family of protocols. It is used as a network layer protocol for routing and host to host packet delivery using an addressing scheme called IP Address(A 32 bit address scheme). IP Protocol also handles a lot of other functions like fragmentation and reassembly of packets to accommodate for transmission of packets over varying sizes of physical channel capacities. It also supports the concept of multicasting and broadcasting (Via IP Options).</p>
<p>In order to come up with a strategy for fuzzing, the first step was to carry out a code walkthrough of relevant functions/APIs and data structures involved in the IPv4 protocol. For that the major files and components studied were:</p>
<ul>
<li>ip_input() => Which carries out the processing of a incoming packet at the network layer for IPv4 (src <a href="https://github.com/NetBSD/src/blob/trunk/sys/netinet/ip_input.c">here</a>)</li>
<li>ip_output() => Which carries out the processing of an outgoing packet at the network layer for IPv4 (src <a href="https://github.com/NetBSD/src/blob/trunk/sys/netinet/ip_output.c">here</a>)</li>
<li>struct ip => Represents the IP header (src <a href="https://github.com/NetBSD/src/blob/trunk/sys/netinet/ip.h">here</a>)</li>
</ul>
<p>These sections of code represent the working of the input and output processing paths of IPv4 protocol and the struct ip is the main IPv4 header. On top of that other APIs related to mbuf (The NetBSD packet), ip_forward(), IP assembly and fragmentation etc. were also studied in order to determine information about packet structure that could be followed.</p>
<p>In order to be able to reach these various aspects of the protocol and be able to fuzz it, we went forward with packet creation that took care of basic fields of the IP Header so that it would not get rejected in trivial cases as mentioned before. Hence we went ahead and fixed these fields:</p>
<ul>
<li><em>IP Version</em>: Set it to 0x4 which is a 4 bit value.</li>
<li><em>IP Header Len</em>: Which is set to a value greater than or equal to sizeof(struct ip). Setting this to greater than that allows for IP Options processing.</li>
<li><em>IP Len</em>: Set it to the size of the random buffer passed by fuzzer.</li>
<li><em>IP Checksum</em>: We calculate the correct checksum for the packet using the internet checksum algorithm.</li>
</ul>
<p>Other fields were allowed to be populated randomly by fuzzer input. Here is an illustration of the IPv4 header with the fields marked in red as fixed.</p>
<p>
<img src="//netbsd.org/~kamil/gsoc_2020/rumpnetfuzz.png">
<p>The packet creation code lies in the following section inside [<a href="https://github.com/NJnisarg/fuzznetrump/blob/master/src/helpers/pkt_create.c#L152">pkt_create.c</a>]. Another important component is the network configuration [located here <a href="https://github.com/NJnisarg/fuzznetrump/blob/master/src/helpers/net_config.c#L173">net_config</a>] where the code related to configuring a TUN/TAP device is present. All the code uses the rumpkernel exposed APIs and syscalls (prepended with rump_sys_) so as to utilize the rumpkernel while executing the application binary. After packet creation and network config is handled the main fuzzing function is written where a series of steps are followed:</p>
<ol>
<li>We call rump_init() to initialize the rumpkernel linked via libraries</li>
<li>We setup the Client and server IP addresses</li>
<li>We setup the TUN device by calling the network config functions described above</li>
<li>We create the packet using the packet creation function utilizing the random buffer passed by the fuzzer and transforming that into a semi-random buffer.</li>
<li>Pass this forged packet into the network stack of the rumpkernel linked with the application binary by calling rump_sys_write on the TUN device setup.</li>
</ol>
<p><strong>IPv6:</strong></p>
<p>IPv6 stands for the Internet protocol version 4. It is the successor of the IPv4 protocol. It came into existence in order to overcome the addressing requirements that could not fit in a 32 bit IPv4 address. It is used as a network layer protocol for routing and host to host packet delivery using an addressing scheme called IPv6 Address(A 128 bit address scheme). It also supports almost similar other functions as IPv4 except some things like fragmentation, broadcast(instead uses multicast). </p>
<p>In order to be able to reach these various aspects of the protocol and be able to fuzz it, we went forward with packet creation that took care of basic fields of the IP Header so that it would not get rejected in trivial cases as mentioned before. Hence we went ahead and fixed these fields:</p>
<ul>
<li><em>IP Version</em>: Set it to 0x6 which is a 4 bit value.</li>
<li><em>IP Hop Limit</em>: This is an alias for TTL. Set it to a maximum possible value of 255(8 bits).</li>
</ul>
<p>Other fields were allowed to be populated randomly by fuzzer input. Allowing the payload len value to be randomly populated allowed processing of various “next headers” or ”Extension headers”. Extension headers carry optional Internet Layer information, and are placed between the fixed header and the upper-layer protocol header. The headers form a chain, using the Next Header fields. The Next Header field in the fixed header indicates the type of the first extension header; the Next Header field of the last extension header indicates the type of the upper-layer protocol header in the payload of the packet. A further work can be done to set the value of the next header chain and form packets for multiple scenarios with a combination of various next headers.</p>
<p><strong>UDP:</strong></p>
<p>UDP stands for User Datagram Protocol. It is one of the simplest protocols and is designed to be simple so that it simply carries payload with minimal overhead. It does not have many options except for checksum information and ports in order to demultiplex the packet to the processes. </p>
<p>Since UDP runs at the transport layer and hence is wrapped up in an IP header. Since we do not want to fuzz the IP code section, we form a well formed IP header so that the packet does not get rejected in the IP processing section. We only randomize the UDP header using the fuzzer input. We used previously built out IP packet creation utilities to form the IP header and then use the fuzzer input for UDP header. </p>
<p>In UDP, we fix the following fields:</p>
<ul>
<li><em>UDP Checksum</em>: Set it to zero in order to avoid checksums.</li>
</ul>
<p><strong>ICMP:</strong></p>
<p>ICMP stands for Internet control message protocol. This protocol is sometimes called a sister protocol of IP protocol and is used as a troubleshooting protocol at the network layer. It is used for major 2 purposes:</p>
<ol>
<li>Error messages</li>
<li>Request-Reply Queries.</li>
</ol>
<p>ICMP has a lot of options and is quite generic in the sense that it handles a lot of error messages and queries. Although ICMP is generally considered at the network layer, it is actually wrapped inside an IP header, hence it has its own protocol number(= 1). Again similar to UDP, we wrap the ICMP headers inside IP headers, hence we do not randomize the IP header and only the ICMP headers using fuzzer input.</p>
<p>In order to test various ICMP messages and queries, we could not fix values for the <em>type </em>and <em>code</em> fields in the ICMP header since they decide the ICMP message type. Also if we allowed random input, most of the packets would get rejected since the number of options of type and code fields are limited and most other values would discard the packet while processing. Hence we came up with a solution where we <em>deterministically</em> modified the input bits from the fuzzer corresponding to the <em>code</em> and <em>type</em> fields. For the <em>type</em> field we simply took a modulo of the number of types(ICMP_NTYPES macro used here). For the value of <em>code </em>, we had to fix values in a certain range based on the <em>type </em>value set already. This technique allowed us to cover all different ICMP message types via the fuzzer input. We also ensured that the input buffer was not modified completely randomly, since that is a bad practice for a feedback-driven fuzzer like ours. Apart from this we fixed the ICMP Checksum field as well by calculating the checksum using the internet checksum algorithm.</p>
<p><strong>Ethernet:</strong></p>
<p>Ethernet protocol defined by the IEEE 802.3 standard is a widely used data link layer protocol. The ethernet packet called a frame carries an IP(or the network layer protocol) datagram. The header is simple with Link Layer Addresses called MAC address (used for switching at data link layer which is a part of addressing), for source and destination each of 6 octets(=48 bytes) present, followed by a 4 octet Ethertype and QTag field. This is followed by payload and finally the FCS(frame check sequence) which is a four-octet cyclic redundancy check (CRC) that allows detection of corrupted data within the entire frame as received on the receiver side. </p>
<p>In case of Ethernet protocol fuzzing, we had to use a TAP device instead of a TUN device, since the TUN device supports passing an IP packet to the network stack, whereas a TAP device accepts an ethernet frame. </p>
<p>For packet creation, we set the source and destination MAC address and let the payload and ethertype be randomly populated by the fuzzer.</p>
<p><br /><br /></p>
<p><strong>Current Progress and Next steps</strong></p>
<p>The project currently has reached a stage where many major internet family protocols have been covered for fuzzing. As described above a structured approach to fuzzing them have been taken by forming packets based on the internal workings of the protocols. Also as mentioned in the previous post, Rumpkernel environment is being used for fuzzing all these protocols. In order to get better results as compared to raw fuzzing, we have taken these steps. In the next report we shall talk about and compare the coverage of raw fuzzing with our approach.</p>
<p>For the next phase of GSoC, the major focus would be to validate this process of fuzzing by various methods to check the penetration of packets into the network stack as well as the code coverage. Also the code would be made more streamlined and standardized so that it can be extended for adding more protocols even beyond the scope of the GSoC project.</p>https://blog.netbsd.org/tnf/entry/gsoc_2020_second_evaluation_reportGSoC 2020 Second Evaluation Report: Curses Library Automated TestingKamil Rytarowski2020-08-07T11:21:00+00:002020-08-07T11:47:52+00:00<i>This report was prepared by Naman Jain as a part of Google Summer of Code 2020</i>
<p>My GSoC project under NetBSD involves the development of test framework of curses library. This blog report is second in series of blog reports; you can have a look at the <a href="https://blog.netbsd.org/tnf/entry/gsoc_reports_curses_library_automated" rel="nofollow">first report</a>. This report would cover the progress made in second coding phase along with providing some insights into the libcurses.</p><i>This report was prepared by Naman Jain as a part of Google Summer of Code 2020</i>
<p>My GSoC project under NetBSD involves the development of test framework of curses library. This blog report is second in series of blog reports; you can have a look at the <a href="https://blog.netbsd.org/tnf/entry/gsoc_reports_curses_library_automated" rel="nofollow">first report</a>. This report would cover the progress made in second coding phase along with providing some insights into the libcurses.</p>
<h2><a id="user-content-complex-characters" class="anchor" aria-hidden="true" href="#complex-characters"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Complex characters</h2>
<p>A complex character is a set of associated character, which may include a spacing character and non-spacing characters associated with it. Typical effects of non-spacing character on associated complex character <em>c</em> include: modifying the appearance of <em>c</em> (like adding diacritical marks) or bridge <em>c</em> with the following character.
The <strong>cchar_t</strong> data type represents a complex character and its rendition. In NetBSD, this data type has following structure:</p>
<div class="highlight highlight-source-c"><pre><span class="pl-k">struct</span> <span class="pl-c1">cchar_t</span> {
<span class="pl-c1">attr_t</span> attributes; <span class="pl-c"><span class="pl-c">/*</span> character attributes <span class="pl-c">*/</span></span>
<span class="pl-k">unsigned</span> elements; <span class="pl-c"><span class="pl-c">/*</span> number of wide char in vals<span class="pl-c">*/</span></span>
<span class="pl-c1">wchar_t</span> vals[CURSES_CCHAR_MAX]; <span class="pl-c"><span class="pl-c">/*</span> wide chars including non-spacing <span class="pl-c">*/</span></span>
};</pre></div>
<p><em>vals</em> array contains the spacing character and associated non-spacing characters. Note that NetBSD supports <strong>wchar_t</strong> (wide character) due to which multi-byte characters are supported. To use the complex characters one has to correctly set the locale settings.
In this coding period, I wrote tests for routines involving complex characters.</p>
<h2><a id="user-content-alternate-character-set" class="anchor" aria-hidden="true" href="#alternate-character-set"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Alternate character set</h2>
<p>When you print "BSD", you would send the hex-codes 42, 53, 44 to the terminal. Capability of graphic capable printers was limited by 8-bit ASCII code. To solve this, additional character sets were introduced. We can switch between the modes using escape sequence. One such character set for Special Graphics is used by curses for line drawing. In a shell you can type</p>
<div class="highlight highlight-source-shell"><pre><span class="pl-c1">echo</span> -e <span class="pl-s"><span class="pl-pds">"</span>\e(0j\e(b<span class="pl-pds">"</span></span></pre></div>
<p>to get a lower-right corner glyph. This enables alternate character mode (<code>\e(</code>), prints a character(<code>j</code>) and disables alternate character mode (<code>\e(b</code>). One might wonder where this 'j' to 'Lower Right Corner glyph' comes from. You may see that mapping ("acsc=``aaffggiijjkkllmmnnooppqqrrssttuuvvwwxxyyzz{{||}}~~,) via</p>
<div class="highlight highlight-source-shell"><pre>infocmp -1 <span class="pl-smi">$TERM</span> <span class="pl-k">|</span> grep acsc</pre></div>
<p>These characters are used in <code>box_set()</code>, <code>border_set()</code>, etc. functions which I tested in the second coding period.</p>
<h2><a id="user-content-progress-in-the-second-coding-phase" class="anchor" aria-hidden="true" href="#progress-in-the-second-coding-phase"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Progress in the second coding phase</h2>
<h3><a id="user-content-improvements-in-the-framework" class="anchor" aria-hidden="true" href="#improvements-in-the-framework"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Improvements in the framework:</h3>
<ol>
<li>Added support for testing of functions to be called before <code>initscr()</code></li>
<li>Updated the unsupported function definitions with some minor bug fixes.</li>
</ol>
<h3><a id="user-content-testing-and-bug-reports" class="anchor" aria-hidden="true" href="#testing-and-bug-reports"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Testing and bug reports</h3>
<ol>
<li>Added tests for following families of functions:
<ul>
<li>Complex character routines.</li>
<li>Line/box drawing routines.</li>
<li>Pad routines.</li>
<li>Window and sub-window operations.</li>
<li>Curson manipulation routines</li>
</ul>
</li>
<li>Reported bugs (and possible fixes if I know):
<ul>
<li><a href="https://gnats.netbsd.org/cgi-bin/query-pr-single.pl?number=55454" rel="nofollow">lib/55454</a> <code>wredrawln()</code> in libcurses does not follow the sensible behaviour [<em>fixed</em>]</li>
<li><a href="https://gnats.netbsd.org/cgi-bin/query-pr-single.pl?number=55460" rel="nofollow">lib/55460</a> copy error in libcurses [<em>fixed</em>]</li>
<li><a href="https://gnats.netbsd.org/cgi-bin/query-pr-single.pl?number=55474" rel="nofollow">lib/55474</a> <code>wattroff()</code> unsets all attributes if passed STANDOUT as argument [<em>standard is not clear, so decided to have as it is</em>]</li>
<li><a href="https://gnats.netbsd.org/cgi-bin/query-pr-single.pl?number=55482" rel="nofollow">lib/55482</a> <code>slk_restore()</code> does not restore the slk screen</li>
<li><a href="https://gnats.netbsd.org/cgi-bin/query-pr-single.pl?number=55484" rel="nofollow">lib/55484</a> <code>newwin()</code> results into seg fault [<em>fixed</em>]</li>
<li><a href="https://gnats.netbsd.org/cgi-bin/query-pr-single.pl?number=55496" rel="nofollow">lib/55496</a> <code>bkgrnd()</code> doesn't work as expected</li>
<li><a href="https://gnats.netbsd.org/cgi-bin/query-pr-single.pl?number=55517" rel="nofollow">lib/55517</a> <code>wresize()</code> function incorrectly resizes the subwindows</li>
</ul>
</li>
</ol>
<p>I would like to thank my mentors Brett and Martin, as well as the NetBSD community for their support whenever I faced some issues.</p>https://blog.netbsd.org/tnf/entry/gsoc_reports_fuzzing_rumpkernel_syscalls1GSoC Reports: Fuzzing Rumpkernel Syscalls, Part 2Kamil Rytarowski2020-08-05T08:42:37+00:002020-08-05T08:42:37+00:00<i>This report was prepared by Aditya Vardhan Padala as a part of Google Summer of Code 2020</i>
<p>I have been working on Fuzzing Rumpkernel Syscalls. This blogpost details the work I have done during my second coding period.</p><i>This report was prepared by Aditya Vardhan Padala as a part of Google Summer of Code 2020</i>
<p>I have been working on Fuzzing Rumpkernel Syscalls. This blogpost details the work I have done during my second coding period.</p>
<h2 id="reproducing-crash-found-in-ioctl-">Reproducing crash found in ioctl()</h2>
<p>Kamil has worked on reproducing the following crash </p>
<pre><code class="lang-bash=">Thread <span class="hljs-number">1</span> <span class="hljs-string">""</span> received signal SIGSEGV, Segmentation fault.
pipe_ioctl (fp=<optimized <span class="hljs-keyword">out</span>>, cmd=<optimized <span class="hljs-keyword">out</span>>, data=<span class="hljs-number">0x7f7fffccd700</span>)
at /usr/src/<span class="hljs-class"><span class="hljs-keyword">lib</span>/<span class="hljs-title">librump</span>/../../<span class="hljs-title">sys</span>/<span class="hljs-title">rump</span>/../<span class="hljs-title">kern</span>/<span class="hljs-title">sys_pipe</span>.<span class="hljs-title">c</span>:1108</span>
<span class="hljs-symbol">warning:</span> Source file is more recent than executable.
<span class="hljs-number">1108</span> *(int *)data = pipe->pipe_buffer.cnt;
(gdb) bt
<span class="hljs-comment">#0 pipe_ioctl (fp=<optimized out>, cmd=<optimized out>, data=0x7f7fffccd700)</span>
at /usr/src/<span class="hljs-class"><span class="hljs-keyword">lib</span>/<span class="hljs-title">librump</span>/../../<span class="hljs-title">sys</span>/<span class="hljs-title">rump</span>/../<span class="hljs-title">kern</span>/<span class="hljs-title">sys_pipe</span>.<span class="hljs-title">c</span>:1108</span>
<span class="hljs-comment">#1 0x000075b0de65083f in sys_ioctl (l=<optimized out>, uap=0x7f7fffccd820, retval=<optimized out>)</span>
at /usr/src/<span class="hljs-class"><span class="hljs-keyword">lib</span>/<span class="hljs-title">librump</span>/../../<span class="hljs-title">sys</span>/<span class="hljs-title">rump</span>/../<span class="hljs-title">kern</span>/<span class="hljs-title">sys_generic</span>.<span class="hljs-title">c</span>:671</span>
<span class="hljs-comment">#2 0x000075b0de6b8957 in sy_call (rval=0x7f7fffccd810, uap=0x7f7fffccd820, l=0x75b0de126500, </span>
sy=<optimized <span class="hljs-keyword">out</span>>) at /usr/src/<span class="hljs-class"><span class="hljs-keyword">lib</span>/<span class="hljs-title">librump</span>/../../<span class="hljs-title">sys</span>/<span class="hljs-title">rump</span>/../<span class="hljs-title">sys</span>/<span class="hljs-title">syscallvar</span>.<span class="hljs-title">h</span>:65</span>
<span class="hljs-comment">#3 sy_invoke (code=54, rval=0x7f7fffccd810, uap=0x7f7fffccd820, l=0x75b0de126500, sy=<optimized out>)</span>
at /usr/src/<span class="hljs-class"><span class="hljs-keyword">lib</span>/<span class="hljs-title">librump</span>/../../<span class="hljs-title">sys</span>/<span class="hljs-title">rump</span>/../<span class="hljs-title">sys</span>/<span class="hljs-title">syscallvar</span>.<span class="hljs-title">h</span>:94</span>
<span class="hljs-comment">#4 rump_syscall (num=num@entry=54, data=data@entry=0x7f7fffccd820, dlen=dlen@entry=24, </span>
retval=retval@entry=<span class="hljs-number">0x7f7fffccd810</span>)
at /usr/src/<span class="hljs-class"><span class="hljs-keyword">lib</span>/<span class="hljs-title">librump</span>/../../<span class="hljs-title">sys</span>/<span class="hljs-title">rump</span>/<span class="hljs-title">librump</span>/<span class="hljs-title">rumpkern</span>/<span class="hljs-title">rump</span>.<span class="hljs-title">c</span>:769</span>
<span class="hljs-comment">#5 0x000075b0de6ad2ca in rump___sysimpl_ioctl (fd=<optimized out>, com=<optimized out>, </span>
data=<optimized <span class="hljs-keyword">out</span>>) at /usr/src/<span class="hljs-class"><span class="hljs-keyword">lib</span>/<span class="hljs-title">librump</span>/../../<span class="hljs-title">sys</span>/<span class="hljs-title">rump</span>/<span class="hljs-title">librump</span>/<span class="hljs-title">rumpkern</span>/<span class="hljs-title">rump_syscalls</span>.<span class="hljs-title">c</span>:979</span>
<span class="hljs-comment">#6 0x0000000000400bf7 in main (argc=1, argv=0x7f7fffccd8c8) at test.c:15</span>
</code></pre>
<p>in the rump using a fuzzer that uses pip2, dup2 and ioctl syscalls and specific arguments that are known to cause a crash upon which my work develops.</p>
<p><a href="https://github.com/adityavardhanpadala/rumpsyscallfuzz/blob/master/honggfuzz/ioctl/ioctl_fuzz2.c">https://github.com/adityavardhanpadala/rumpsyscallfuzz/blob/master/honggfuzz/ioctl/ioctl_fuzz2.c</a></p>
<p>Since rump is a multithreaded process. Crash occurs in any of those threads. By using a core dump we can quickly investigate the crash and fetch the backtrace from gdb for verification however this is not viable in the long run as you would be loading your working directory with lots of core dumps which consume a lot of space. So we need a better way to reproduce crashes.</p>
<h2 id="crash-reproducers">Crash Reproducers</h2>
<p>Getting crash reproducers working took quite a while. If we look at HF_ITER() function in honggfuzz, it is a simple wrapper for HonggfuzzFetchData() to fetch buffer of fixed size from the fuzzer.</p>
<pre><code class="lang-cpp=">void HonggfuzzFetchData(const uint8<span class="hljs-emphasis">_t** buf_</span>ptr, size<span class="hljs-emphasis">_t* len_</span>ptr) {
<span class="hljs-bullet">.
</span><span class="hljs-bullet">.
</span><span class="hljs-bullet">.
</span><span class="hljs-bullet">.
</span><span class="hljs-strong">*buf_ptr = inputFile;
*</span>len<span class="hljs-emphasis">_ptr = (size_</span>t)rcvLen;
<span class="hljs-bullet">.
</span><span class="hljs-bullet">.
</span>}
</code></pre>
<p>And if we observe the attribute we notice that <code>inputFile</code> is mmaped. </p>
<pre><code class="lang-cpp="><span class="hljs-comment">//libhfuzz/fetch.c:26</span>
<span class="hljs-keyword">if</span> ((inputFile = mmap(<span class="hljs-literal">NULL</span>, _HF_INPUT_MAX_SIZE, PROT_READ, MAP_SHARED, _HF_INPUT_FD, <span class="hljs-number">0</span>)) ==
MAP_FAILED) {
PLOG_F(<span class="hljs-string">"mmap(fd=%d, size=%zu) of the input file failed"</span>, _HF_INPUT_FD,
(<span class="hljs-keyword">size_t</span>)_HF_INPUT_MAX_SIZE);
}
</code></pre>
<p>So in a similar approach HF_ITER() can be modified to read input from a file and be mmapped so that we can reuse the reproducers generated by honggfuzz.</p>
<p>Attempts have been made to use getchar(3) for fetching the buffer via STDIN but for some unknown reason it failed so we switched to mmap(2) </p>
<p>So we overload HF_ITER() function whenever we require to reproduce a crash. I chose the following approach to use the reproducers. So whenever we need to reproduce a crash we just define CRASH_REPR.</p>
<pre><code class="lang-cpp=">
<span class="hljs-function"><span class="hljs-keyword">static</span>
<span class="hljs-keyword">void</span> <span class="hljs-title">Initialize</span><span class="hljs-params">(<span class="hljs-keyword">void</span>)</span>
</span>{
<span class="hljs-meta">#<span class="hljs-meta-keyword">ifdef</span> CRASH_REPR</span>
FILE *fp = fopen(argv[<span class="hljs-number">1</span>], <span class="hljs-string">"r+"</span>);
data = <span class="hljs-built_in">malloc</span>(max_size);
fread(data, max_size, <span class="hljs-number">1</span>, fp);
fclose(fp);
<span class="hljs-meta">#<span class="hljs-meta-keyword">endif</span></span>
<span class="hljs-comment">// Initialise the rumpkernel only once.</span>
<span class="hljs-keyword">if</span>(rump_init() != <span class="hljs-number">0</span>)
__builtin_trap();
}
<span class="hljs-meta">#<span class="hljs-meta-keyword">ifdef</span> CRASH_REPR</span>
<span class="hljs-function"><span class="hljs-keyword">void</span> <span class="hljs-title">HF_ITER</span><span class="hljs-params">(<span class="hljs-keyword">uint8_t</span> **buf, <span class="hljs-keyword">size_t</span> *len)</span> </span>{
*buf = (<span class="hljs-keyword">uint8_t</span> *)data;
*len = max_size;
<span class="hljs-keyword">return</span>;
}
<span class="hljs-meta">#<span class="hljs-meta-keyword">else</span></span>
<span class="hljs-function">EXTERN <span class="hljs-keyword">void</span> <span class="hljs-title">HF_ITER</span><span class="hljs-params">(<span class="hljs-keyword">uint8_t</span> **buf, <span class="hljs-keyword">size_t</span> *len)</span></span>;
<span class="hljs-meta">#<span class="hljs-meta-keyword">endif</span></span>
</code></pre>
<p>This way we can easily reproduce crashes that we get and get the backtraces.</p>
<h2 id="generating-c-reproducers">Generating C reproducers</h2>
<p>Now the main goal is to create a c file which can reproduce the same crash occuring due to the reproducer. This is done by writing all the syscall executions to a file with arguments so they can directly be compiled and used.</p>
<pre><code class="lang-cpp=">#ifdef CRASH_REPR
FILE *fp = fopen(<span class="hljs-string">"/tmp/repro.c"</span>,<span class="hljs-string">"a+"</span>)<span class="hljs-comment">;</span>
fprintf(<span class="hljs-name">fp</span>, <span class="hljs-string">"rump_sys_ioctl(%"</span> PRIu8 <span class="hljs-string">", %"</span> PRIu64 <span class="hljs-string">");\n"</span>,get_u8(),get_ioctl_request())<span class="hljs-comment">;</span>
fclose(<span class="hljs-name">fp</span>)<span class="hljs-comment">;</span>
#else
rump_sys_ioctl(<span class="hljs-name">get_u8</span>(), get_ioctl_request())<span class="hljs-comment">;</span>
#endif
</code></pre>
<p>I followed the same above method for all the syscalls that are executed. So I get a proper order of syscalls executed in native c code that I can simply reuse.</p>
<h3 id="obstacles">Obstacles</h3>
<p>The number of times each syscall is executed before getting to a crash is quite high. So trying to perform a write to a file or STDOUT will create a lot of overhead when the number of syscalls executed are quite high. This method is good enough but a bit of optimization will make it even better.</p>
<h2 id="to-do">To-Do</h2>
<ul>
<li>./build.sh building rump on linux+netbsd</li>
<li>pregenerating fuzzer input using the implementation similar to that used in syzkaller.</li>
</ul>
<p>Finally I thank my mentors Siddharth Muralee, Maciej Grochowski, Christos Zoulas for their guidance and Kamil Rytarowski for his constant support whenever I needed it.</p>https://blog.netbsd.org/tnf/entry/gsoc_reports_enhancing_syzkaller_support1GSoC Reports: Enhancing Syzkaller support for NetBSD, Part 2Kamil Rytarowski2020-08-05T08:10:10+00:002020-08-05T08:10:10+00:00<i>This report was prepared by Ayushi Sharma as a part of Google Summer of Code 2020</i>
<p>As a part of Google summer code 2020, I have been working on <a href="https://wiki.netbsd.org/projects/project/syzkaller/">Enhance the Syzkaller support for NetBSD</a>. This post summarises the work done in the past month.</p>
<p>For work done in the first coding period, you can take a look at the <a href="https://blog.netbsd.org/tnf/entry/gsoc_reports_enhancing_syzkaller_support">previous post</a>.</p>
<h2>Automation for enhancement</h2>
<p>With an aim of increasing the number of syscalls fuzzed, we have decided to automate the addition of descriptions for syscalls as well as ioctl device drivers in a customised way for NetBSD.</p><i>This report was prepared by Ayushi Sharma as a part of Google Summer of Code 2020</i>
<p>As a part of Google summer code 2020, I have been working on <a href="https://wiki.netbsd.org/projects/project/syzkaller/">Enhance the Syzkaller support for NetBSD</a>. This post summarises the work done in the past month.</p>
<p>For work done in the first coding period, you can take a look at the <a href="https://blog.netbsd.org/tnf/entry/gsoc_reports_enhancing_syzkaller_support">previous post</a>.</p>
<h2>Automation for enhancement</h2>
<p>With an aim of increasing the number of syscalls fuzzed, we have decided to automate the addition of descriptions for syscalls as well as ioctl device drivers in a customised way for NetBSD.</p>
<h2>Design</h2>
<p>All the ioctl commands for a device driver in NetBSD are stored inside the <q>/src/sys/dev/<driver_name>/</q> folder. The idea is to get information related to a particular ioctl command by extracting required information from the source code of drivers. To achieve the same, we have broken down our project into majorly three phases.</p>
<ol>
<li>Generating preprocessed files</li>
<li>Extracting information required for generating descriptions</li>
<li>Conversion to syzkaller’s grammar</li>
</ol>
<img src="//netbsd.org/~kamil/gsoc_2020/sys2syz.png" alt="Design" style="border:10px solid white;">
<h2>Generating Preprocessed files</h2>
<p>For a given preprocessed file, c2xml tool outputs the preprocessed C code in xml format. Further, the intermediate xml format descriptions would help us to smoothly transform the c code to syzkaller specific descriptions, in the last stage of this tool. We have used <a href=https://github.com/rizsotto/Bear>Bear</a> as an aid for fetching commands to preprocess files for a particular driver. Bear generates a file called compile_commands.json which stores the commands used for compiling a file in json format. We then run these commands with ‘-E’ gcc flag to fetch the preprocessed files.These preprocessed files then serve as an input to the c2xml program.</p>
<h2>Extractor</h2>
<p>Definition of <a href="//man.NetBSD.org/ioctl.9">ioctl</a> calls defined in header files of device driver in NetBSD can be broken down to:</p>
<img src="//netbsd.org/~kamil/gsoc_2020/ioctl_def.png" alt="ioctl" >
<p>When we see it from syzkaller’s perspective, there are basically three significant parts we need to extract for adding description to syzkaller.</p>
<p>Description of a particular ioctl command acc to syzkaller’s grammar:</p>
<p>ioctl$FOOIOCTL(fd <fd_driver>, cmd const[FOOIOCTL], pt ptr[DIR, <ptr_type>])
<br>
<img src="//netbsd.org/~kamil/gsoc_2020/ioctl_desc_1.png" alt="ioctl description">
<br>
<img src="//netbsd.org/~kamil/gsoc_2020/ioctl_desc_2.png" alt="ioctl description">
<p>These definitions can be grepped from a device’s header files. The type information or description for pointer can then be extracted from the output files generated by c2xml. If the third argument is a struct, the direction of the pointer is determined with the help of fun() macros.
<h2>To-Do</h2>
<p>The extracted descriptions have to be converted into syzkaller-friendly grammer. We plan to add support for syscalls too , which would ease the addition of complex compat syscalls. This would help us to increase the syzkaller’s coverage significantly.
<h2>Stats</h2>
Along with this, We have continued to add support for few more syscalls these include:
<ul>
<li>ksem(2) family</li>
<li>mount(2) family</li>
</ul>
Syscalls related to sockets have also been added. This has increased syscall coverage percentage to 50.35.
<p>Atlast, I would like to thank my mentors - Cryo, Siddharth Muralee and Santhosh along with Kamil for their guidance and support. I am thankful to NetBSD community too along with Google for providing me such an amazing opportunity. https://blog.netbsd.org/tnf/entry/the_gnu_gdb_debugger_and2The GNU GDB Debugger and NetBSD (Part 3)Kamil Rytarowski2020-08-04T16:45:37+00:002020-08-04T16:45:37+00:00The NetBSD team of developers maintains two copies of GDB:
<ul>
<li>One in the base-system with a stack of local patches.</li>
<li>One in pkgsrc with mostly build fix patches.</li>
</ul>
<p>
The base-system version of GDB (GPLv3) still relies on a set of local patches.
I set a goal to reduce the local patches to bare minimum, ideally reaching no local modifications at all.The NetBSD team of developers maintains two copies of GDB:
<ul>
<li>One in the base-system with a stack of local patches.</li>
<li>One in pkgsrc with mostly build fix patches.</li>
</ul>
<p>
The base-system version of GDB (GPLv3) still relies on a set of local patches.
I set a goal to reduce the local patches to bare minimum, ideally reaching no local modifications at all.
<p>
<h1>GDB changes</h1>
<p>
I've written an integration of GDB with fork(2) and vfork(2) events.
Unfortunately, this support (present in a local copy of GDB in the base-system) had not been merged so far, because
there is a generic kernel regression with the pg_jobc variable. This variable can be called
a reference counter of the number of processes within a process group that has a parent with control over a terminal.
The semantics of this variable are not very well defined and in the result the number can become negative.
This unexpected state of pg_jobc resulted in spurious crashes during kernel fuzzing. As a result
new kernel assertions checking for non-negative pg_jobc values were introduced in order to catch the anomalies quickly.
GDB as a ptrace(2)-based application happened to reproduce negative pg_jobc values quickly and reliably and this stopped
the further adoption of the fork(2) and vfork(2) patch in GDB, until the pg_jobc behavior is enhanced.
I was planning to include support for posix_spawn(3) events as well, as they are implemented as a first-class
operation through a syscall, however this is also blocked by the pg_jobc blocker.
<p>
A local patch for GDB is stored
<a href="//netbsd.org/~kamil/patch-00276-gdb-fork-vfork-events.txt">here</a>
for the time being.
<p>
I've enable multi-process mode in the NetBSD native target.
This enabled proper support for multiple inferiors and ptrace(2)
assisted management of the inferior processes and their threads.
<p>
<pre>
(gdb) info inferior
Num Description Connection Executable
* 1 process 14952 1 (native) /usr/bin/dig
2 <null> 1 (native)
3 process 25684 1 (native) /bin/ls
4 <null> 1 (native) /bin/ls
</pre>
<p>
Without this change, additional inferiors could be already added, but not properly controlled.
<p>
I've implemented the xfer_partial TARGET_OBJECT_SIGNAL_INFO support for NetBSD.
NetBSD implements reading and overwriting siginfo_t received by the
tracee. With TARGET_OBJECT_SIGNAL_INFO signal information can be
examined and modified through the special variable $_siginfo.
Currently NetBSD uses an identical siginfo type on
all architectures, so there is no support for architecture-specific fields.
<pre>
(gdb) b main
Breakpoint 1 at 0x71a0
(gdb) r
Starting program: /bin/ps
Breakpoint 1, 0x00000000002071a0 in main ()
(gdb) p $_siginfo
$1 = {si_pad = {5, 0, 0, 0, 1, 0 <repeats 19 times>, 1, 0 <repeats 103 times>}, _info = {_signo = 5,
_code = 1, _errno = 0, _pad = 0, _reason = {_rt = {_pid = 0, _uid = 0, _value = {sival_int = 1,
sival_ptr = 0x1}}, _child = {_pid = 0, _uid = 0, _status = 1, _utime = 0, _stime = 0},
_fault = {_addr = 0x0, _trap = 1, _trap2 = 0, _trap3 = 0}, _poll = {_band = 0, _fd = 1},
_syscall = {_sysnum = 0, _retval = {0, 1}, _error = 0, _args = {0, 0, 0, 0, 0, 0, 0, 0}},
_ptrace_state = {_pe_report_event = 0, _option = {_pe_other_pid = 0, _pe_lwp = 0}}}}}
</pre>
<p>
NetBSD, contrary to Linux and other BSDs, supports a ptrace(2) operation to
generate a core(5) file from a running process. This operation is used in the
base-system gcore(1) program. The gcore functionality is also delivered by GDB, and
I have prepared new code for GDB to wire PT_DUMPCORE into the GDB code for NetBSD,
and thus support GDB's gcore functionality.
This patch is still waiting in upstream review.
A local copy of the patch is
<a href="//netbsd.org/~kamil/patch-00277-gdb-dumpcore.txt">here</a>.
<p>
<pre>
(gdb) r
Starting program: /bin/ps
Breakpoint 1, 0x00000000002071a0 in main ()
(gdb) gcore
Saved corefile core.4378
(gdb) !file core.4378
core.4378: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), NetBSD-style, from 'ps', pid=4378, uid=1000, gid=100, nlwps=1, lwp=4378 (signal 5/code 1)
</pre>
<p>
<h1>Plan for the next milestone</h1>
<p>
Rewrite the gdbserver support and submit upstream.
https://blog.netbsd.org/tnf/entry/gsoc_reports_enhancing_syzkaller_supportGSoC Reports: Enhancing Syzkaller support for NetBSD, Part 1Kamil Rytarowski2020-07-13T22:18:21+00:002020-07-13T22:18:21+00:00<i>This report was prepared by Ayushi Sharma as a part of Google Summer of Code 2020</i>
<p>I have been working on the project - <a href="https://wiki.netbsd.org/projects/project/syzkaller/">Enhance the Syzkaller support for NetBSD</a>, as a part of GSoc’20. Past two months have given me quite an enriching experience, pushing me to comprehend more knowledge on fuzzers. This post would give a peek into the work which has been done so far.</p><i>This report was prepared by Ayushi Sharma as a part of Google Summer of Code 2020</i>
<p>I have been working on the project - <a href="https://wiki.netbsd.org/projects/project/syzkaller/">Enhance the Syzkaller support for NetBSD</a>, as a part of GSoc’20. Past two months have given me quite an enriching experience, pushing me to comprehend more knowledge on fuzzers. This post would give a peek into the work which has been done so far.</p>
<h2>Syzkaller</h2>
<p>Syzkaller is a coverage guided fuzzer developed by Google, to fuzz the system calls mainly keeping linux in mind. However, the support has been extended to fuzz the system calls of other Operating Systems as well for eg. Akaros, FreeBSD, NetBSD, OpenBSD, Windows etc.</p>
<p> An automated system Syzbot continuously runs the syzkaller fuzzer on NetBSD kernel and reports the crashes <p>
<img src="//netbsd.org/~kamil/gsoc_2020/syzbot.png" alt="Syzbot" height="333">
<h2>Increasing syscall support</h2>
<p>Initially, the syscall support for <a href="https://gist.github.com/ais2397/1bed83535b7b65a7d488deb310a0dd78">Linux</a> as well as <a href="https://gist.github.com/ais2397/b8886661c698be83fcc757858a2184b4">FreeBSD</a> was analysed by an automated script. Also <a href="https://gist.github.com/ais2397/b6a737b9ad0836301194478549e733c8">coverage of NetBSD syscalls</a> was looked over. This helped us to easily port a few syscalls descriptions for NetBSD. The necessary tweaks were made according to the documentation which describes rules for writing syscall descriptions.</p>
<p>Major groups of syscalls which have been added:
<ul>
<li>statfs</li>
<li>__getlogin
<li>getsid</li>
<li>mknod</li>
<li>utimes</li>
<li>wait4</li>
<li>seek</li>
<li>setitimer</li>
<li>setpriority</li>
<li>getrusage</li>
<li>clock_settime</li>
<li>nanosleep</li>
<li>getdents </li>
<li>acct </li>
<li>dup</li>
</ul>
</p>
<h2>Bugs Found</h2>
There were a few bugs reported as a result of adding the descriptions for syscalls of the mentioned syscall families. Few of them are yet to be fixed.
<ul>
<li><a href="https://syzkaller.appspot.com/bug?id=e70e6b077e9dac9807c86fd0aa418aa837180598"> compat_20_getfstat</a></li>
<li><a href="https://syzkaller.appspot.com/bug?id=ed011128934a1410452457bcebfe634fcf84b9a2">clock_settime50</a></li>
<li><a href="https://syzkaller.appspot.com/bug?id=6d4983b443f3b9ec458313601e468b9cd4bf781e">ioctl</a></li>
</ul>
<h2>Stats</h2>
<p>Syscall coverage percent for NetBSD has now increased from nearly 30 to 44.96. Addition of compat syscalls resulted in getting a few new bugs.</p>
<p> Percentage of syscalls covered in few of the other Operating Systems:
<ul>
<li>Linux: 82</li>
<li>FreeBSD: 37</li>
<li>OpenBSD: 61</li>
</ul>
</p>
<h2>Conclusion</h2>
<p>In the next phase I would be working on generating the syscall descriptions using automation and adding ioctl device drivers with it’s help.</p>
<p>Atlast, I would like to thank my mentors Cryo, Siddharth and Santhosh for their constant support and guidance.I am also thankful to NetBSD community for being kind to give me this opportunity of having an amazing summer this year.</p>https://blog.netbsd.org/tnf/entry/gsoc_reports_extending_the_functionalityGSoC Reports: Extending the functionality of NetPGP, Part 1Kamil Rytarowski2020-07-13T22:05:18+00:002020-07-13T22:20:52+00:00<i>This report was prepared by Jason High as a part of Google Summer of Code 2020</i>
<p>NetPGP is a library and suite of tools implementing OpenPGP under a BSD license. As part of Google Summer of Code 2020, we are working to extend its functionality and work towards greater parity with similar tools. During the first phase, we have made the following contributions</p>
<ol>
<li>Added the Blowfish block cipher</li>
<li>ECDSA key creation</li>
<li>ECDSA signature and verification</li>
<li>Symmetric file encryption/decryption</li>
<li>S2K Iterated+Salt for symmetric encryption</li>
</ol><i>This report was prepared by Jason High as a part of Google Summer of Code 2020</i>
<p>NetPGP is a library and suite of tools implementing OpenPGP under a BSD license. As part of Google Summer of Code 2020, we are working to extend its functionality and work towards greater parity with similar tools. During the first phase, we have made the following contributions</p>
<ol>
<li>Added the Blowfish block cipher</li>
<li>ECDSA key creation</li>
<li>ECDSA signature and verification</li>
<li>Symmetric file encryption/decryption</li>
<li>S2K Iterated+Salt for symmetric encryption</li>
</ol>
<p>ECDSA key generation is done using the '--ecdsa' flag with netpgpkeys or the 'ecdsa' property if using libnetpgp</p>
<pre><code>[jhigh@gsoc2020nb gsoc]$ netpgpkeys --generate-key --ecdsa --homedir=/tmp
signature secp521r1/ECDSA a0cdb04e3e8c5e34 2020-06-25
fingerprint d9e0 2ae5 1d2f a9ae eb96 ebd4 a0cd b04e 3e8c 5e34
uid jhigh@localhost
Enter passphrase for a0cdb04e3e8c5e34:
Repeat passphrase for a0cdb04e3e8c5e34:
generated keys in directory /tmp/a0cdb04e3e8c5e34
[jhigh@gsoc2020nb gsoc]$
[jhigh@gsoc2020nb gsoc]$ ls -l /tmp/a0cdb04e3e8c5e34
total 16
-rw------- 1 jhigh wheel 331 Jun 25 16:03 pubring.gpg
-rw------- 1 jhigh wheel 440 Jun 25 16:03 secring.gpg
[jhigh@gsoc2020nb gsoc]$
</code></pre>
<p>Signing with ECDSA does not require any changes</p>
<pre><code>[jhigh@gsoc2020nb gsoc]$ netpgp --sign --homedir=/tmp/a0cdb04e3e8c5e34 --detach --armor testfile.txt
signature secp521r1/ECDSA a0cdb04e3e8c5e34 2020-06-25
fingerprint d9e0 2ae5 1d2f a9ae eb96 ebd4 a0cd b04e 3e8c 5e34
uid jhigh@localhost
netpgp passphrase:
[jhigh@gsoc2020nb gsoc]$
[jhigh@gsoc2020nb gsoc]$ cat testfile.txt.asc
-----BEGIN PGP MESSAGE-----
wqcEABMCABYFAl71EYwFAwAAAAAJEKDNsE4+jF40AAAVPgIJASyzuZgyS13FHHF/9qk6E3pYra2H
tDdkqxYzNIqKnWHaB+a4J+/R7FkZItbC/EyXH5YA68AC1dJ7tRN/tJNIWfYjAgUb75SvM2mLHk13
qmBo48S0Ai8C82G4nT7/16VF2OOUn7F/3XICghoQOyS1nxJilj8u2uphLOoy9VayL1ErORIZVw==
=p30e
-----END PGP MESSAGE-----
[jhigh@gsoc2020nb gsoc]$
</code></pre>
<p>Verification remains the same, as well.</p>
<pre><code>[jhigh@gsoc2020nb gsoc]$ netpgp --homedir=/tmp/a0cdb04e3e8c5e34 --verify testfile.txt.asc
netpgp: assuming signed data in "testfile.txt"
Good signature for testfile.txt.asc made Thu Jun 25 16:05:16 2020
using ECDSA key a0cdb04e3e8c5e34
signature secp521r1/ECDSA a0cdb04e3e8c5e34 2020-06-25
fingerprint d9e0 2ae5 1d2f a9ae eb96 ebd4 a0cd b04e 3e8c 5e34
uid jhigh@localhost
[jhigh@gsoc2020nb gsoc]$
</code></pre>
<p>Symmetric encryption is now possible using the '--symmetric' flag with netpgp or the 'symmetric' property in libnetpgp</p>
<pre><code>[jhigh@gsoc2020nb gsoc]$ netpgp --encrypt --symmetric --armor testfile.txt
Enter passphrase:
Repeat passphrase:
[jhigh@gsoc2020nb gsoc]$
[jhigh@gsoc2020nb gsoc]$ cat testfile.txt.asc
-----BEGIN PGP MESSAGE-----
wwwEAwEIc39k1V6xVi3SPwEl2ww75Ufjhw7UA0gO/niahHWK50DFHSD1lB10nUyCTgRLe6iS9QZl
av5Nji9BuQFcrqo03I1jG/L9s/4hww==
=x41O
-----END PGP MESSAGE-----
[jhigh@gsoc2020nb gsoc]$
</code></pre>
<p>Decryption of symmetric packets requires no changes</p>
<pre><code>[jhigh@gsoc2020nb gsoc]$ netpgp --decrypt testfile.txt.asc
netpgp passphrase:
[jhigh@gsoc2020nb gsoc]$
</code></pre>
<p>We added two new flags to support s2k mode 3: '--s2k-mode' and '--s2k-count'. See RFC4880 for details.</p>
<pre><code>[jhigh@gsoc2020nb gsoc]$ netpgp --encrypt --symmetric --s2k-mode=3 --s2k-count=96 testfile.txt
Enter passphrase:
Repeat passphrase:
[jhigh@gsoc2020nb gsoc]$
[jhigh@gsoc2020nb gsoc]$ gpg -d testfile.txt.gpg
gpg: CAST5 encrypted data
gpg: encrypted with 1 passphrase
this
is
a
test
[jhigh@gsoc2020nb gsoc]$
</code></pre>https://blog.netbsd.org/tnf/entry/gsoc_reports_curses_library_automatedGSoC Reports: Curses Library Automated Testing, Part 1Kamil Rytarowski2020-07-13T21:40:13+00:002020-07-13T22:21:34+00:00<i>This report was prepared by Naman Jain as a part of Google Summer of Code 2020</i>
<h2><a id="user-content-introduction" class="anchor" aria-hidden="true" href="#introduction"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Introduction</h2>
<p>My GSoC project under NetBSD involves the development of test framework of curses library. Automated Test Framework (ATF) was introduced in 2007 but ATF cannot be used directly for curses testing for several reasons most important of them being curses has functions which do timed reads/writes which is hard to do with just piping characters to test applications. Also, stdin is not a tty device and behaves differently and may affect the results. A lot of work regarding this has been done and we have a separate test framework in place for testing curses.</p>
<p>The aim of project is to build a robust test suite for the library and complete the SUSv2 specification. This includes writing tests for the remaining functions and enhancing the existing ones. Meanwhile, the support for complex character function has to be completed along with fixing some bugs, adding features and improving the test framework.</p><i>This report was prepared by Naman Jain as a part of Google Summer of Code 2020</i>
<h2><a id="user-content-introduction" class="anchor" aria-hidden="true" href="#introduction"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Introduction</h2>
<p>My GSoC project under NetBSD involves the development of test framework of curses library. Automated Test Framework (ATF) was introduced in 2007 but ATF cannot be used directly for curses testing for several reasons most important of them being curses has functions which do timed reads/writes which is hard to do with just piping characters to test applications. Also, stdin is not a tty device and behaves differently and may affect the results. A lot of work regarding this has been done and we have a separate test framework in place for testing curses.</p>
<p>The aim of project is to build a robust test suite for the library and complete the SUSv2 specification. This includes writing tests for the remaining functions and enhancing the existing ones. Meanwhile, the support for complex character function has to be completed along with fixing some bugs, adding features and improving the test framework.</p>
<h2><a id="user-content-why-did-i-chose-this-project" class="anchor" aria-hidden="true" href="#why-did-i-chose-this-project"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Why did I chose this project?</h2>
<p>I am a final year undergraduate at Indian Institute of Technology, Kanpur. I have my majors in Computer Science and Engineering, and I am specifically interested in algorithms and computer systems. I had worked on building and testing a library on Distributed Tracing at an internship and understand the usefulness of having a test suite in place. Libcurses being very special in itself for the testing purpose interested me. Knowing some of the concepts of compiler design made my interest a bit more profound.</p>
<h2><a id="user-content-test-framwork" class="anchor" aria-hidden="true" href="#test-framwork"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Test Framwork</h2>
<p>The testframework consists of 2 programs, director and slave. The framework provides its own simple language for writing tests. The slave is a curses application capable of running any curses function, while the director acts as a coordinator and interprets the test file and drives the slave program. The director can also capture slave's output which can be used for comparison with desired output.</p>
<p><a target="_blank" rel="noopener noreferrer" href="//netbsd.org/~kamil/gsoc_2020/director-slave.jpeg"><img src="//netbsd.org/~kamil/gsoc_2020/director-slave.jpeg" alt="slave-director model" title="Slave-Director model" style="max-width:100%;"></a></p>
<p>The director forks a process operating in pty and executes a slave program on that fork. The master side of pty is used for getting the data stream that the curses function call produces which can be futher used to check the correctness of behaviour. Director and slave communicate via pipes; command pipe and slave pipe. The command pipe carries the function name and arguments, while slave pipe carries return codes and values from function calls.</p>
<p>Let's walk through a sample test to understand how this works. Consider a sample program:</p>
<pre><code>include start
call win newwin 2 5 2 5
check win NON_NULL
call OK waddstr $win "Hello World!"
call OK wrefresh $win
compare waddstr_refresh.chk
</code></pre>
<p>This is a basic program which initialises the screen, creates new window, checks if the window creation was successful, adds as string "Hello World!" on the window, refreshes the window, and compares it with desired output stored in check file. The details of the language can be accessed at <a href="https://github.com/NetBSD/src/blob/trunk/tests/lib/libcurses/testframe.txt">libcurses testframe</a>.</p>
<p>The test file is interpreted by the language parser and the correponding actions are taken. Let's look how line #2 is processed. This command creates a window using <code>newwin()</code>. The line is ultimately parsed as <a href="https://github.com/NetBSD/src/blob/1d30657e76c9400c15eb4e4cfcdff2fea6c65a5a/tests/lib/libcurses/director/testlang_parse.y#L237"><code>call: CALL result fn_name args eol</code></a> grammar rule and executes the function <a href="https://github.com/NetBSD/src/blob/1d30657e76c9400c15eb4e4cfcdff2fea6c65a5a/tests/lib/libcurses/director/testlang_parse.y#L1035"><code>do_funtion_call()</code></a>). Now, this function <a href="https://github.com/NetBSD/src/blob/1d30657e76c9400c15eb4e4cfcdff2fea6c65a5a/tests/lib/libcurses/director/testlang_parse.y#L1048">sends</a> function name and arguments using command pipe to the slave. The slave, who is waiting to get command from the director, reads from the pipe and <a href="https://github.com/NetBSD/src/blob/1d30657e76c9400c15eb4e4cfcdff2fea6c65a5a/tests/lib/libcurses/slave/slave.c#L144">executes</a> the command. This executes the correponding curses function from the <a href="https://github.com/NetBSD/src/blob/1d30657e76c9400c15eb4e4cfcdff2fea6c65a5a/tests/lib/libcurses/slave/command_table.h#L40">command table</a> and the pointer to new window is <a href="https://github.com/NetBSD/src/blob/1d30657e76c9400c15eb4e4cfcdff2fea6c65a5a/tests/lib/libcurses/slave/curses_commands.c#L3582">returned</a> via the slave pipe (<a href="https://github.com/NetBSD/src/blob/1d30657e76c9400c15eb4e4cfcdff2fea6c65a5a/tests/lib/libcurses/slave/commands.c#L167">here</a>) after passing wrappers of functions. The director <a href="https://github.com/NetBSD/src/blob/1d30657e76c9400c15eb4e4cfcdff2fea6c65a5a/tests/lib/libcurses/director/testlang_parse.y#L1140">recieves</a> them, and returned value is assigned to a variable(<code>win</code> in line#2) or compared (<code>OK</code> in line#4). This is the typical life cycle of a certain function call made in tests.</p>
<p>Along with these, the test framework provides capability to <code>include</code> other test (line#1), <code>check</code> the variable content (line#3), <code>compare</code> the data stream due to function call in pty with desired stream (line#6). Tester can also provide inputs to functions via <code>input</code> directive, perform delay via <code>delay</code> directive, assign values to variables via <code>assign</code> directive, and create a wide or complex charater via <code>wchar</code> and <code>cchar</code> directives respectively. The framework supports 3 kind of strings; null terminated string, byte string, and string of type chtype, based on the quotes enclosing it.</p>
<h2><a id="user-content-progress-till-the-first-evaluation" class="anchor" aria-hidden="true" href="#progress-till-the-first-evaluation"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Progress till the first evaluation</h2>
<h3><a id="user-content-improvements-in-the-framework" class="anchor" aria-hidden="true" href="#improvements-in-the-framework"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Improvements in the framework:</h3>
<ol>
<li>Automated the checkfile generation that has to be done manually earlier.</li>
<li>Completed the support for complex chacter tests in director and slave.</li>
<li>Added features like variable-variable comparison.</li>
<li>Fixed non-critical bugs in the framework.</li>
<li>Refactored the code.</li>
</ol>
<h3><a id="user-content-testing-and-bug-reports" class="anchor" aria-hidden="true" href="#testing-and-bug-reports"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Testing and bug reports</h3>
<ol>
<li>Wrote tests for wide character routines.</li>
<li>Reported bugs (and possible fixes if I know):
<ul>
<li><a href="https://gnats.netbsd.org/cgi-bin/query-pr-single.pl?number=55433" rel="nofollow">lib/55433</a> Bug in special character handling of ins_wstr() of libcurses</li>
<li><a href="https://gnats.netbsd.org/cgi-bin/query-pr-single.pl?number=55434" rel="nofollow">lib/55434</a> Bug in hline() in libcurses [fixed]</li>
<li><a href="https://gnats.netbsd.org/cgi-bin/query-pr-single.pl?number=55443" rel="nofollow">lib/55443</a> setcchar() incorrectly sets the number of elements in cchar [fixed]</li>
</ul>
</li>
</ol>
<h2><a id="user-content-project-proposal-and-references" class="anchor" aria-hidden="true" href="#project-proposal-and-references"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Project Proposal and References:</h2>
<ul>
<li>Proposal: <a href="https://github.com/NamanJain8/curses/blob/master/reports/proposal.pdf">https://github.com/NamanJain8/curses/blob/master/reports/proposal.pdf</a></li>
<li>Project Repo: <a href="https://github.com/NamanJain8/curses">https://github.com/NamanJain8/curses</a></li>
<li>Test Language Details: <a href="https://github.com/NetBSD/src/blob/trunk/tests/lib/libcurses/testframe.txt">https://github.com/NetBSD/src/blob/trunk/tests/lib/libcurses/testframe.txt</a></li>
<li>Wonderful report by Brett Lymn: <a href="https://github.com/NamanJain8/curses/blob/master/reports/curses-testframe.pdf">https://github.com/NamanJain8/curses/blob/master/reports/curses-testframe.pdf</a></li>
</ul>https://blog.netbsd.org/tnf/entry/gsoc_reports_fuzzing_the_netbsdGSoC Reports: Fuzzing the NetBSD Network Stack in a Rumpkernel Environment, Part 1Kamil Rytarowski2020-07-13T14:58:58+00:002020-07-13T22:22:24+00:00<i>This report was prepared by Nisarg Joshi as a part of Google Summer of Code 2020</i>
<p><strong>Introduction:</strong></p>
<p>The objective of this project is to fuzz the various protocols and layers of the network stack of NetBSD using rumpkernel. This project is being carried out as a part of GSoC 2020. This blog post is regarding the project, the concepts and tools involved, the objectives and the current progress and next steps.</p>
<p><strong>Fuzzing:</strong></p>
<p>Fuzzing or fuzz testing is an automated software testing technique in which a program is tested by passing unusual, unexpected or semi-random input generated data to the input of the program and repeatedly doing so, trying to crash the program and detect potential bugs or undealt corner cases.</p>
<p>There are several tools available today that enable this which are known as fuzzers. An effective fuzzer generates semi-valid inputs that are "valid enough" in that they are not directly rejected by the parser, but do create unexpected behaviors deeper in the program and are "invalid enough" to expose corner cases that have not been properly dealt with. </p>
<p>Fuzzers can be of various types like dumb vs smart, generation-based vs mutation-based and so on. A dumb fuzzer generates random input without looking at the input format or model but it can follow some sophisticated algorithms like in AFL, though considered a dumb fuzzer as it just flips bits and replaces bytes, still uses a genetic algorithm to create new test cases, where as a smart fuzzer will follow an input model to generate semi-random data that can penetrate well in the code and trigger more edge cases. Mutation and generation fuzzers handle test case generation differently. Mutation fuzzers mutate a supplied seed input object, while generation fuzzers generate new test cases from a supplied model.</p>
<p>Some examples of popular fuzzers are: AFL(American Fuzzy Lop), Syzkaller, Honggfuzz.</p><i>This report was prepared by Nisarg Joshi as a part of Google Summer of Code 2020</i>
<p><strong>Introduction:</strong></p>
<p>The objective of this project is to fuzz the various protocols and layers of the network stack of NetBSD using rumpkernel. This project is being carried out as a part of GSoC 2020. This blog post is regarding the project, the concepts and tools involved, the objectives and the current progress and next steps.</p>
<p><strong>Fuzzing:</strong></p>
<p>Fuzzing or fuzz testing is an automated software testing technique in which a program is tested by passing unusual, unexpected or semi-random input generated data to the input of the program and repeatedly doing so, trying to crash the program and detect potential bugs or undealt corner cases.</p>
<p>There are several tools available today that enable this which are known as fuzzers. An effective fuzzer generates semi-valid inputs that are "valid enough" in that they are not directly rejected by the parser, but do create unexpected behaviors deeper in the program and are "invalid enough" to expose corner cases that have not been properly dealt with. </p>
<p>Fuzzers can be of various types like dumb vs smart, generation-based vs mutation-based and so on. A dumb fuzzer generates random input without looking at the input format or model but it can follow some sophisticated algorithms like in AFL, though considered a dumb fuzzer as it just flips bits and replaces bytes, still uses a genetic algorithm to create new test cases, where as a smart fuzzer will follow an input model to generate semi-random data that can penetrate well in the code and trigger more edge cases. Mutation and generation fuzzers handle test case generation differently. Mutation fuzzers mutate a supplied seed input object, while generation fuzzers generate new test cases from a supplied model.</p>
<p>Some examples of popular fuzzers are: AFL(American Fuzzy Lop), Syzkaller, Honggfuzz.</p>
<p><strong>RumpKernel</strong></p>
<p>Kernels can have several different architectures like monolithic, microkernel, exokernel etc. An interesting architecture is the “anykernel” architecture, according to wikipedia, “The "anykernel" concept refers to an architecture-agnostic approach to drivers where drivers can either be compiled into the monolithic kernel or be run as a userspace process, microkernel-style, without code changes.” Rumpkernel is an implementation of this anykernel architecture. In case of the NetBSD rumpkernel, all the NetBSD subsystems like file system, network stack, drivers etc are compiled into standalone libraries that can be linked into any application process to utilize the functionalities of the kernel, like the file system or the drivers. This allows us to run and test various components of NetBSD kernel as a library linked to programs running in the user space.</p>
<p>This idea of rumpkernel is really helpful in fuzzing the components of the kernel. We can fuzz separate subsystems and allow it to crash without having to manage the crash of a running Operating System. Also the fact that context switching has an overhead in case syscalls are made to the kernel, Rumpkernel running in the userspace can eliminate this and save time.(Also since the spectre-meltdown vulnerabilities, system calls have become more costly due to the security reasons)</p>
<p><strong>HonggFuzz + Rumpkernel Network Stack:</strong></p>
<p>In this project we will use the outlined Rumpkernel’s network stack and a fuzzer called the honggfuzz. Honggfuzz is a security oriented, feedback-driven, evolutionary, easy-to-use fuzzer. It is maintained by google.(<a href="https://github.com/google/honggfuzz">https://github.com/google/honggfuzz</a>)</p>
<p>The project is hosted on github at: <a href="https://github.com/NJnisarg/fuzznetrump/">https://github.com/NJnisarg/fuzznetrump/</a> .The Readme can help in setting it up. We first require NetBSD installed either on a physical machine or on a virtual machine like qemu. Then we will build the NetBSD distribution by grabbing the latest sources(<a href="https://github.com/NetBSD/src">https://github.com/NetBSD/src</a>). We will enable fuzzer coverage by using MKSANITIZER and MKLIBCSANITIZER flags and using the ASan(Address) and UBSan(Undefined Behavior) sanitizers. These sanitizers will help the fuzzer in catching bugs related to undefined behavior and address and memory leaks. After that we will grab one of the fuzzer programs(example: src/hfuzz_ip_output_fuzz.c) and chroot into the newly built distribution from the host NetBSD OS. Then we will compile it using hfuzz-clang by linking the required rumpkernel libraries (for example in our case: rump, rumpnet, rumpnet_netinet and so on). This is where we use the rumpkernel as libraries linked to our program. The program will access the network stack of the linked rumpkernel and the fuzzer will fuzz those components of the kernel. The compiled binary will be run with honggfuzz. Detailed steps are outlined in the Readme of the linked repository.</p>
<p><strong>Our Approach for network stack fuzzing:</strong></p>
<p>We have planned to fuzz various protocols at different layers of the TCP/IP stack. We have started with mostly widely used yet simple protocols like IP(v4), UDP etc. Along the progress of the project, we will be adding support for more L3(and above) protocols like ICMP, IP(v6), TCP as well as L2 protocols like Ethernet as a bit later phase.</p>
<p>The network stack has 2 paths: </p>
<ol>
<li>Input/ingress path</li>
<li>Output/egress path</li>
</ol>
<p>A packet is sent down the network stack via a socket from an application from the output path, whereas a packet is said to be received on a network interface into the network stack via the input path. Each network protocol has major input and output APIs for the packet processing. Example IP protocol has an ip_input() function to process an incoming packet and an ip_output() function to process an outgoing packet. We are planning to fuzz each protocol’s output and input APIs by sending the packet via the output path and receiving the packet via input path respectively. </p>
<p>In order to fuzz the output and input path, the network stack setup and configuration we have is as follows:</p>
<ul>
<li>We have a TUN device to which we can read and write a packet.</li>
<li>We have a socket that is bound to the TUN device, which can send and receive packets</li>
<li>In order to fuzz the input path, we “write” a packet to the TUN interface, which emulates a received packet on the network stack.</li>
<li>In order to fuzz the output path, we send a packet via the socket to the TUN interface to fuzz the output path.</li>
</ul>
<p>For carrying out all the above setup, we have separated out the common function for creating and configuring the TUN device and socket into a file called “net_config.c”</p>
<p>Also in order to reduce the rejection of packets carrying random data for trivial cases like checksum or header len, we have created functions that create/forge semi-random packets using the input data from honggfuzz. We manipulate certain fields in the packet to ensure that it does not get rejected trivially by the network stack and hence can reach and trigger deeper parts of the code. These functions for packet creations are located in the “pkt_create.c” file. For each protocol we fuzz, we add these functions to forge the headers and the packet. Currently we have support from UDP and IP(v4).</p>
<p>With these building blocks we have written programs like hfuzz_ip_output_fuzz.c, hfuzz_ip_input_fuzz.c etc, which setup the TUN device and socket using net_config.c and after taking the random data from honggfuzz, use it to forge a packet and send it down or up the stack. We compile these programs using hfuzz-clang as mentioned above and run it under honggfuzz to fuzz the network stack’s particular APIs.</p>
<p><strong>Current Progress:</strong></p>
<p>Following things were worked upon in the first phase:</p>
<ul>
<li>Getting honggfuzz functional for NetBSD(thanks to Kamil for those patches)</li>
<li>Coming up with the strategy for network configuration and packet creation. Writing utilities for the same.</li>
<li>Adding fuzzing code for protocols like IP(v4) and UDP.</li>
<li>Carrying out fuzzing for those protocols.</li>
</ul>
<p><strong>Next Steps:</strong></p>
<p>As next steps following things are planned for upcoming phase:</p>
<ul>
<li>Making changes and improvements by taking suggestions from the mentors.</li>
<li>Adding support for ICMP, IP(v6), TCP and later on for Ethernet.</li>
<li>Analyze and come up with effective ways to improve the fuzzing by focusing on the packet creation part.</li>
<li>Standardize the code to be extensible for adding future protocols.</li>
</ul>https://blog.netbsd.org/tnf/entry/gsoc_reports_make_system_3GSoC Reports: Make system(3) and popen(3) use posix_spawn(3) internally, Part 1Kamil Rytarowski2020-07-13T14:11:35+00:002022-02-24T10:48:15+00:00<i>This report was prepared by Nikita Ronja Gillmann as a part of Google Summer of Code 2020</i>
<p>This is my first report for the Google Summer of Code project I am working on for NetBSD.</p><p>Prior work: In GSoC 2012 Charles Zhang <a href="https://blog.netbsd.org/tnf/entry/posix_spawn_syscall_added">added the posix_spawn syscall</a>
which according to <a href="https://web.archive.org/web/20150905210045/http://netbsd-soc.sourceforge.net/projects/posix_spawn/">its SF repository</a> at the time (maybe even now, I have not looked very
much into comparing all other systems and libcs + kernels) is an in-kernel implementation of posix_spawn which provides performance benefits compared
to FreeBSD and other systems which had a userspace implementation (in 2012).</p><p>After 1 week of reading POSIX and writing code, 2 weeks of coding and another
1.5 weeks of bugfixes I have successfully implemented posix_spawn in usage
in system(3) and popen(3) internally.</p><p>The biggest challenge for me was to understand POSIX, to read the standard.
I am used to reading more formal books, but I can't remember working with the
posix standard directly before.</p><p>The next part of my Google Summer of Code project will focus on similar rewrites
of NetBSD's sh(1).</p><i>This report was prepared by Nikita Ronja Gillmann as a part of Google Summer of Code 2020</i>
<p>This is my first report for the Google Summer of Code project I am working on for NetBSD.</p><p>Prior work: In GSoC 2012 Charles Zhang <a href="https://blog.netbsd.org/tnf/entry/posix_spawn_syscall_added">added the posix_spawn syscall</a>
which according to <a href="https://web.archive.org/web/20150905210045/http://netbsd-soc.sourceforge.net/projects/posix_spawn/">its SF repository</a> at the time (maybe even now, I have not looked very
much into comparing all other systems and libcs + kernels) is an in-kernel implementation of posix_spawn which provides performance benefits compared
to FreeBSD and other systems which had a userspace implementation (in 2012).</p><p>After 1 week of reading POSIX and writing code, 2 weeks of coding and another
1.5 weeks of bugfixes I have successfully implemented posix_spawn in usage
in system(3) and popen(3) internally.</p><p>The biggest challenge for me was to understand POSIX, to read the standard.
I am used to reading more formal books, but I can't remember working with the
posix standard directly before.</p><p>The next part of my Google Summer of Code project will focus on similar rewrites
of NetBSD's sh(1).</p><h2>system(3)</h2><p>The prototype</p><blockquote><p>int system(const char *command);</p></blockquote><p>remains the same. Below I'm just commenting on the differences, not the whole
function, and therefore only include code block where the versions differ the
most. The full work can be found at <a href="https://github.com/nikicoon/gsoc2020">gsoc2020</a>
as well as <a href="https://github.com/nikicoon/src">src</a> and will be submitted for inclusion
later in the project.</p><p>Previously we'd use vfork, sigaction, and execve in this stdlib function.</p><p>The biggest difference to the 2015 version of our system version is
the usage of posix_spawnattr_ where we'd use sigaction before, and posix_spawn
where execve executes the command in a vfork'd child:</p><blockquote><pre><code> posix_spawnattr_init(&attr);
posix_spawnattr_setsigmask(&attr, &omask);
posix_spawnattr_setflags(&attr, POSIX_SPAWN_SETSIGDEF|POSIX_SPAWN_SETSIGMASK);
(void)__readlockenv();
status = posix_spawn(&pid, _PATH_BSHELL, NULL, &attr, __UNCONST(argp), environ);
(void)__unlockenv();
posix_spawnattr_destroy(&attr);</code></pre></blockquote><p><a href="//netbsd.org/~nikita/gsoc2020/system.c">The full version can be found here.</a></p><p>The prototype of posix_spawn is:</p><blockquote><p>int posix_spawn(pid_t *restrict pid, const char *restrict path,
const posix_spawn_file_actions_t *file_actions,
const posix_spawnattr_t *restrict attrp,
char *const argv[restrict], char *const envp[restrict]);</p></blockquote><p>We first initialize a spawn attributes object with the default value for all
of its attributes set.
A spawn attributes object is used to modify the behavior of posix_spawn().</p><p>The previous fork-exec switch included calls to sigaction to set the
behavior associated with SIGINT and SIGQUIT as defined by POSIX:</p><blockquote><p>The system() function shall ignore the SIGINT and SIGQUIT signals, and shall block the SIGCHLD signal, while waiting for the command to terminate. If this might cause the application to miss a signal that would have killed it, then the application should examine the return value from system() and take whatever action is appropriate to the application if the command terminated due to receipt of a signal.
source: https://pubs.opengroup.org/onlinepubs/9699919799/functions/system.html</p></blockquote><p>This has been achieved with a combination of posix_spawnattr_setsigmask() and
posix_spawnattr_setflags() in the initialized attributes object referenced by
attr.</p><p>As before we call __readlockenv() and then call posix_spawn() which returns the
process ID of the child process in the variable pointed to by 'pid', and returns
zero as the function return value.</p><p>The old code:</p><blockquote><pre><code> (void)__readlockenv();
switch(pid = vfork()) {
case -1: /* error */
(void)__unlockenv();
sigaction(SIGINT, &intsa, NULL);
sigaction(SIGQUIT, &quitsa, NULL);
(void)sigprocmask(SIG_SETMASK, &omask, NULL);
return -1;
case 0: /* child */
sigaction(SIGINT, &intsa, NULL);
sigaction(SIGQUIT, &quitsa, NULL);
(void)sigprocmask(SIG_SETMASK, &omask, NULL);
execve(_PATH_BSHELL, __UNCONST(argp), environ);
_exit(127);
}
(void)__unlockenv();</code></pre></blockquote><h2>popen(3), popenve(3)</h2><p>As with system, the prototype of both functions remains the same:</p><blockquote><p>FILE * popenve(const char *cmd, char *const *argv, char *const *envp, const char *type);
FILE * popen(const char *cmd, const char *type);</p></blockquote><p>Since the popen and popenve implementation in NetBSD's libc use
a couple of shared helper functions, I was able to change both
functions while keeping the majority of the changes focused on
(some of ) the helper functions (pdes_child).</p><p>pdes_child, an internal function in popen.c, now takes one more argument
(const char *cmd) for the command to pass to posix_spawn which is called
in pdes_child.</p><p>pdes_child previously looked like this:</p><pre><code>static void
pdes_child(int *pdes, const char *type)
{
struct pid *old;
/* POSIX.2 B.3.2.2 "popen() shall ensure that any streams
from previous popen() calls that remain open in the
parent process are closed in the new child process. */
for (old = pidlist; old; old = old->next)
#ifdef _REENTRANT
(void)close(old->fd); /* don't allow a flush */
#else
(void)close(fileno(old->fp)); /* don't allow a flush */
#endif
if (type[0] == 'r') {
(void)close(pdes[0]);
if (pdes[1] != STDOUT_FILENO) {
(void)dup2(pdes[1], STDOUT_FILENO);
(void)close(pdes[1]);
}
if (type[1] == '+')
(void)dup2(STDOUT_FILENO, STDIN_FILENO);
} else {
(void)close(pdes[1]);
if (pdes[0] != STDIN_FILENO) {
(void)dup2(pdes[0], STDIN_FILENO);
(void)close(pdes[0]);
}
}
}</code></pre><p>This is the new version
(<a href="//netbsd.org/~nikita/gsoc2020/popen.c">the whole file is here</a>):</p><pre><code>static int
pdes_child(int *pdes, const char *type, const char *cmd)
{
struct pid *old;
posix_spawn_file_actions_t file_action_obj;
pid_t pid;
const char *argp[] = {"sh", "-c", NULL, NULL};
argp[2] = cmd;
int error;
error = posix_spawn_file_actions_init(&file_action_obj);
if (error) {
goto fail;
}
/* POSIX.2 B.3.2.2 "popen() shall ensure that any streams
from previous popen() calls that remain open in the
parent process are closed in the new child process. */
for (old = pidlist; old; old = old->next)
#ifdef _REENTRANT
error = posix_spawn_file_actions_addclose(&file_action_obj, old->fd); /* don't allow a flush */
if (error) {
goto fail;
}
#else
error = posix_spawn_file_actions_addclose(&file_action_obj, fileno(old->fp)); /* don't allow a flush */
if (error) {
goto fail;
}
#endif
if (type[0] == 'r') {
error = posix_spawn_file_actions_addclose(&file_action_obj, pdes[0]);
if (error) {
goto fail;
}
if (pdes[1] != STDOUT_FILENO) {
error = posix_spawn_file_actions_adddup2(&file_action_obj, pdes[1], STDOUT_FILENO);
if (error) {
goto fail;
}
error = posix_spawn_file_actions_addclose(&file_action_obj, pdes[1]);
if (error) {
goto fail;
}
}
if (type[1] == '+') {
error = posix_spawn_file_actions_adddup2(&file_action_obj, STDOUT_FILENO, STDIN_FILENO);
if (error) {
goto fail;
}
}
} else {
error = posix_spawn_file_actions_addclose(&file_action_obj, pdes[1]);
if (error) {
goto fail;
}
if (pdes[0] != STDIN_FILENO) {
error = posix_spawn_file_actions_adddup2(&file_action_obj, pdes[0], STDIN_FILENO);
if (error) {
goto fail;
}
error = posix_spawn_file_actions_addclose(&file_action_obj, pdes[0]);
if (error) {
goto fail;
}
}
}
(void)__readlockenv();
error = posix_spawn(&pid, _PATH_BSHELL, &file_action_obj, 0, __UNCONST(argp), environ);
if (error) {
(void)__unlockenv();
goto fail;
}
(void)__unlockenv();
error = posix_spawn_file_actions_destroy(&file_action_obj);
/*
* TODO: if _destroy() fails we have to go on, otherwise we
* leak the pid.
*/
if (error) {
errno = error;
return -1;
}
return pid;
fail:
errno = error;
posix_spawn_file_actions_destroy(&file_action_obj);
return -1;
}</code></pre><p>On a high level what happens in pdes_child() and popen is that we first lock the
pidlist_mutex. Then we create a file file action list for all concurrent
popen() / popenve() instances and the side of the pipe not necessary, and the
move to stdin/stdout. We unlock the pidlist_mutex. Finally we return the list
and destroy.</p><p>In the new version of this helper function which now handles the majority of
what popen/popenve did, we have to initialize a file_actions object which by
default contains no file actions for posix_spawn() to perform. Since we have
to have error handling and a common return value for the functions calling
pdes_child() and deconstruction, we make use of goto in some parts of this
function.</p><p>The close() and dup2() actions now get replaced by corresponding file_actions
syscalls, they are used to specify a series of actions to be performed by
a posix_spawn operation.</p><p>After this series of actions, we call _readlockenv(), and call posix_spawn with
the file_action object and the other arguments to be executed. If it succeeds,
we return the pid of the child to popen, otherwise we return -1, in both cases
we destroy the file_action object before we proceed.</p><p>In popen and popenve our code has been reduced to just the 'pid == -1' branch,
everything else happens in pdes_child() now.</p><p>After readlockenv we call pdes_child and pass it the command to execute in the
posix_spawn'd child process; if pdes_child returns -1 we run the old error
handling code. Likewise for popenve.</p><p>popen, old:</p><pre><code>FILE *
popen(const char *cmd, const char *type)
{
struct pid *cur;
int pdes[2], serrno;
pid_t pid;
_DIAGASSERT(cmd != NULL);
_DIAGASSERT(type != NULL);
if ((cur = pdes_get(pdes, &type)) == NULL)
return NULL;
MUTEX_LOCK();
(void)__readlockenv();
switch (pid = vfork()) {
case -1: /* Error. */
serrno = errno;
(void)__unlockenv();
MUTEX_UNLOCK();
pdes_error(pdes, cur);
errno = serrno;
return NULL;
/* NOTREACHED */
case 0: /* Child. */
pdes_child(pdes, type);
execl(_PATH_BSHELL, "sh", "-c", cmd, NULL);
_exit(127);
/* NOTREACHED */
}
(void)__unlockenv();
pdes_parent(pdes, cur, pid, type);
MUTEX_UNLOCK();
return cur->fp;
}</code></pre><p>popen, new:</p><pre><code>FILE *
popen(const char *cmd, const char *type)
{
struct pid *cur;
int pdes[2], serrno;
pid_t pid;
_DIAGASSERT(cmd != NULL);
_DIAGASSERT(type != NULL);
if ((cur = pdes_get(pdes, &type)) == NULL)
return NULL;
MUTEX_LOCK();
(void)__readlockenv();
pid = pdes_child(pdes, type, cmd);
if (pid == -1) {
/* Error. */
serrno = errno;
(void)__unlockenv();
MUTEX_UNLOCK();
pdes_error(pdes, cur);
errno = serrno;
return NULL;
/* NOTREACHED */
}
(void)__unlockenv();
pdes_parent(pdes, cur, pid, type);
MUTEX_UNLOCK();
return cur->fp;
}</code></pre><p>popenve, old:</p><pre><code>FILE *
popenve(const char *cmd, char *const *argv, char *const *envp, const char *type)
{
struct pid *cur;
int pdes[2], serrno;
pid_t pid;
_DIAGASSERT(cmd != NULL);
_DIAGASSERT(type != NULL);
if ((cur = pdes_get(pdes, &type)) == NULL)
return NULL;
MUTEX_LOCK();
switch (pid = vfork()) {
case -1: /* Error. */
serrno = errno;
MUTEX_UNLOCK();
pdes_error(pdes, cur);
errno = serrno;
return NULL;
/* NOTREACHED */
case 0: /* Child. */
pdes_child(pdes, type);
execve(cmd, argv, envp);
_exit(127);
/* NOTREACHED */
}
pdes_parent(pdes, cur, pid, type);
MUTEX_UNLOCK();
return cur->fp;
}</code></pre><p>popenve, new:</p><pre><code>FILE *
popenve(const char *cmd, char *const *argv, char *const *envp, const char *type)
{
struct pid *cur;
int pdes[2], serrno;
pid_t pid;
_DIAGASSERT(cmd != NULL);
_DIAGASSERT(type != NULL);
if ((cur = pdes_get(pdes, &type)) == NULL)
return NULL;
MUTEX_LOCK();
pid = pdes_child(pdes, type, cmd);
if (pid == -1) {
/* Error. */
serrno = errno;
MUTEX_UNLOCK();
pdes_error(pdes, cur);
errno = serrno;
return NULL;
/* NOTREACHED */
}
pdes_parent(pdes, cur, pid, type);
MUTEX_UNLOCK();
return cur->fp;
}</code></pre>https://blog.netbsd.org/tnf/entry/gsoc_reports_fuzzing_rumpkernel_syscallsGSoC Reports: Fuzzing Rumpkernel Syscalls, Part 1Kamil Rytarowski2020-07-13T13:08:58+00:002020-07-13T22:23:43+00:00<i>This report was prepared by Aditya Vardhan Padala as a part of Google Summer of Code 2020</i>
<p>
<p>It has been a great opportunity to contribute to NetBSD as a part of Google Summer Of Code '20. The aim of the project I am working on is to setup a proper environment to fuzz the rumpkernel syscalls. This is the first report on the progress made so far.</p>
<p>Rumpkernels provide all the necessary components to run applications on baremetal without the necessity of an operating system. Simply put it is way to run kernel code in user space.</p>
<p>The main goal of rumpkernels in netbsd is to run,debug,examine and develop kernel drivers as easy as possible in the user space without having to run the entire kernel but run the exact same kernel code in userspace. This makes most of the components(drivers) easily portable to different environments.</p>
<p>Rump Kernels are constructed out of components, So the drivers are built as libraries and these libraries are linked to an interface(some application) that makes use of the libraries(drivers). So we need not build the entire monolithic kernel just the required parts of the kernel.</p><i>This report was prepared by Aditya Vardhan Padala as a part of Google Summer of Code 2020</i>
<p>
<p>It has been a great opportunity to contribute to NetBSD as a part of Google Summer Of Code '20. The aim of the project I am working on is to setup a proper environment to fuzz the rumpkernel syscalls. This is the first report on the progress made so far.</p>
<p>Rumpkernels provide all the necessary components to run applications on baremetal without the necessity of an operating system. Simply put it is way to run kernel code in user space.</p>
<p>The main goal of rumpkernels in netbsd is to run,debug,examine and develop kernel drivers as easy as possible in the user space without having to run the entire kernel but run the exact same kernel code in userspace. This makes most of the components(drivers) easily portable to different environments.</p>
<p>Rump Kernels are constructed out of components, So the drivers are built as libraries and these libraries are linked to an interface(some application) that makes use of the libraries(drivers). So we need not build the entire monolithic kernel just the required parts of the kernel.</p>
<h2 id="why-honggfuzz-">Why Honggfuzz?</h2>
<p>I considered Honggfuzz the best place to start with for fuzzing the syscall layer as suggested by my mentors. LibFuzzer style library fuzzing method helped me in exploring how syscalls are implemented in the rumpkernel. With LibFuzzer we have the flexibility of modifying a few in-kernel functions as per the requirements to best suit the fuzzing target.</p>
<h2 id="fuzzing-target">Fuzzing target</h2>
<p>Taking a close look at <code>src/sys/rump/librump/rumpkern/rump_syscalls.c</code>
we observe that this is where the rump syscalls are defined. These functions are responsible for creating the arguments structure (like wrappers) and passing it to syscalls which is define</p>
<pre><code class="lang-cpp="><span class="hljs-function"><span class="hljs-title">rsys_syscall</span><span class="hljs-params">(num, data, dlen, retval)</span></span>
</code></pre>
<p> which is defined from </p>
<pre><code class="lang-cpp="><span class="hljs-function"><span class="hljs-title">rump_syscall</span><span class="hljs-params">(num, data, dlen, retval)</span></span>
</code></pre>
<p>This function is the one that invokes the execution of the syscalls. So this should be the target for fuzzing syscalls.</p>
<h2 id="fuzzing-using-honggfuzz">Fuzzing Using Honggfuzz</h2>
<p>Initially we used the classic LibFuzzer style.</p>
<pre><code class="lang-cpp=">
<span class="hljs-function"><span class="hljs-keyword">int</span>
<span class="hljs-title">LLVMFuzzerTestOneInput</span><span class="hljs-params">(<span class="hljs-keyword">const</span> <span class="hljs-keyword">uint8_t</span> *Data, <span class="hljs-keyword">size_t</span> Size)</span>
</span>{
<span class="hljs-keyword">if</span>(Size != <span class="hljs-number">2</span> + <span class="hljs-number">8</span>+<span class="hljs-keyword">sizeof</span>(<span class="hljs-keyword">uint8_t</span>))
<span class="hljs-keyword">return</span> <span class="hljs-number">0</span>;
ExecuteSyscallusingtheData();
<span class="hljs-keyword">return</span> <span class="hljs-number">0</span>;
}
</code></pre>
<p>However this approach into issues when we had to overload <code>copyin()</code>, <code>copyout()</code>, <code>copyinstr()</code>, <code>copyoutstr()</code> functions as the pointers that is used in these functions are from the Data buffer that the fuzzer provides for each fuzz iteration.</p>
<pre><code class="lang-cpp="><span class="hljs-function"><span class="hljs-keyword">int</span>
<span class="hljs-title">copyin</span><span class="hljs-params">(<span class="hljs-keyword">const</span> <span class="hljs-keyword">void</span> *uaddr, <span class="hljs-keyword">void</span> *kaddr, <span class="hljs-keyword">size_t</span> len)</span>
</span>{
..
..
..
..
<span class="hljs-keyword">if</span> (RUMP_LOCALPROC_P(curproc)) {
<span class="hljs-built_in">memcpy</span>(kaddr, uaddr, len); <- slow
} <span class="hljs-keyword">else</span> <span class="hljs-keyword">if</span> (len) {
error = rump_sysproxy_copyin(RUMP_SPVM2CTL(curproc->p_vmspace),
uaddr, kaddr, len);
}
<span class="hljs-keyword">return</span> error;
}
</code></pre>
<p>Honggfuzz provides us HF_ITER interface which will be useful to actively fetch inputs from the fuzzer for example.</p>
<pre><code class="lang-cpp="><span class="hljs-function"><span class="hljs-keyword">extern</span> <span class="hljs-keyword">void</span> <span class="hljs-title">HF_ITER</span><span class="hljs-params">(<span class="hljs-keyword">uint8_t</span> **buf, <span class="hljs-keyword">size_t</span> *len)</span></span>;
<span class="hljs-function"><span class="hljs-keyword">int</span>
<span class="hljs-title">main</span><span class="hljs-params">()</span>
</span>{
<span class="hljs-keyword">for</span> (;;) {
<span class="hljs-keyword">uint8_t</span> *buf;
<span class="hljs-keyword">size_t</span> len;
HF_ITER(&buf, &len);
DoSomethingWithInput(buf, len);
}
<span class="hljs-keyword">return</span> <span class="hljs-number">0</span>;
}
</code></pre>
<p>So we switched to a faster method of using <code>HF_ITER()</code> in honggfuzz to fetch the data from the fuzzer as it is relatively flexible to use.</p>
<pre><code class="lang-cpp="><span class="hljs-function">EXTERN <span class="hljs-keyword">int</span>
<span class="hljs-title">rumpns_copyin</span><span class="hljs-params">(<span class="hljs-keyword">const</span> <span class="hljs-keyword">void</span> *uaddr, <span class="hljs-keyword">void</span> *kaddr, <span class="hljs-keyword">size_t</span> len)</span>
</span>{
<span class="hljs-keyword">int</span> error = <span class="hljs-number">0</span>;
<span class="hljs-keyword">if</span> (len == <span class="hljs-number">0</span>)
<span class="hljs-keyword">return</span> <span class="hljs-number">0</span>;
<span class="hljs-comment">//HF_MEMGET() is a wrapper around HF_ITER()</span>
HF_MEMGET(kaddr, len);
<span class="hljs-keyword">return</span> error;
}
</code></pre>
<p>Similar overloading is done for <code>copyout()</code>, <code>copyinstr()</code>, <code>copyoutstr()</code>.</p>
<p>The current efforts to fuzz the rump syscalls using honggfuzz can be found <a href="https://github.com/adityavardhanpadala/rumpsyscallfuzz">here</a>.</p>
<p>This gave quite a speed bump to the "dumb" fuzzer from few tens iterations to couple of hundreds as we replaced the memcpys with a <a href="https://github.com/adityavardhanpadala/rumpsyscallfuzz/blob/master/honggfuzz/syscall_fuzzv2.c#L183">wrapper</a> around HF_ITER().</p>
<p><img src="//netbsd.org/~kamil/gsoc_2020/rump1.png" alt=""></p>
<h3 id="further-work-by-kamil-rytarowski">Further work by Kamil Rytarowski</h3>
<p>Kamil has detected that overloading copyin()/copyout() functions have a shortcoming that we are mangling internal functionality of the rump that uses this copying mechanism, especially in rump_init(), but also in other rump wrappers, e.g. opening a file with rump_sys_open().</p>
<p>The decision has been made to alter the fuzzing mechanism from pumping random honggfuzz assisted data into the rump-kernel APIs and intercept copyin()/copyount() family of functions to add prior knowledge of the arguments that are valid. The initial target set by Kamil was to reproduce the following rump kernel crash (first detected with syzkaller in the real kernel):</p>
<pre><code class="lang-cpp=">
<span class="hljs-meta">#<span class="hljs-meta-keyword">include</span> <span class="hljs-meta-string"><sys/types.h></span></span>
<span class="hljs-meta">#<span class="hljs-meta-keyword">include</span> <span class="hljs-meta-string"><sys/ioctl.h></span></span>
<span class="hljs-meta">#<span class="hljs-meta-keyword">include</span> <span class="hljs-meta-string"><rump/rump.h></span></span>
<span class="hljs-meta">#<span class="hljs-meta-keyword">include</span> <span class="hljs-meta-string"><rump/rump_syscalls.h></span></span>
<span class="hljs-built_in">int</span>
main(<span class="hljs-built_in">int</span> argc, char **argv)
{
<span class="hljs-built_in">int</span> filedes[<span class="hljs-number">2</span>]<span class="hljs-comment">;</span>
rump_init()<span class="hljs-comment">;</span>
rump_sys_pipe2(filedes, <span class="hljs-number">0</span>)<span class="hljs-comment">;</span>
rump_sys_dup2(filedes[<span class="hljs-number">1</span>], filedes[<span class="hljs-number">0</span>])<span class="hljs-comment">;</span>
rump_sys_ioctl(filedes[<span class="hljs-number">1</span>], FIONWRITE)<span class="hljs-comment">;</span>
}
</code></pre>
<p><a href="https://www.netbsd.org/~kamil/panic/rump_panic.c">https://www.netbsd.org/~kamil/panic/rump_panic.c</a></p>
<p>We can compare that panicking the real kernel was analogous to the rump calls:</p>
<pre><code class="lang-cpp="><span class="hljs-meta">#<span class="hljs-meta-keyword">include</span> <span class="hljs-meta-string"><sys/types.h></span></span>
<span class="hljs-meta">#<span class="hljs-meta-keyword">include</span> <span class="hljs-meta-string"><sys/ioctl.h></span></span>
<span class="hljs-meta">#<span class="hljs-meta-keyword">include</span> <span class="hljs-meta-string"><unistd.h></span></span>
<span class="hljs-meta">#<span class="hljs-meta-keyword">include</span> <span class="hljs-meta-string"><stdio.h></span></span>
<span class="hljs-meta">#<span class="hljs-meta-keyword">include</span> <span class="hljs-meta-string"><stdlib.h></span></span>
<span class="hljs-meta">#<span class="hljs-meta-keyword">include</span> <span class="hljs-meta-string"><err.h></span></span>
<span class="hljs-meta">#<span class="hljs-meta-keyword">include</span> <span class="hljs-meta-string"><fcntl.h></span></span>
<span class="hljs-built_in">int</span>
main(<span class="hljs-built_in">int</span> argc, char **argv)
{
<span class="hljs-built_in">int</span> filedes[<span class="hljs-number">2</span>]<span class="hljs-comment">;</span>
pipe2(filedes, <span class="hljs-number">0</span>)<span class="hljs-comment">;</span>
dup2(filedes[<span class="hljs-number">1</span>], filedes[<span class="hljs-number">0</span>])<span class="hljs-comment">;</span>
ioctl(filedes[<span class="hljs-number">1</span>], FIONWRITE)<span class="hljs-comment">;</span>
<span class="hljs-keyword">return</span> <span class="hljs-number">0</span><span class="hljs-comment">;</span>
}
</code></pre>
<p><a href="https://www.netbsd.org/~kamil/panic/panic.c">https://www.netbsd.org/~kamil/panic/panic.c</a></p>
<p>Thus, Kamil wrote a new fuzzing program that uses pip2, dup2 and ioctl APIs exclusively and has prior knowledge about valid arguments to these functions (as the reproducer is using perfectly valid NetBSD syscalls and arguments):</p>
<p><a href="https://github.com/adityavardhanpadala/rumpsyscallfuzz/blob/master/honggfuzz/ioctl/ioctl_fuzz2.c">https://github.com/adityavardhanpadala/rumpsyscallfuzz/blob/master/honggfuzz/ioctl/ioctl_fuzz2.c</a></p>
<p>The code instead of checking out impossible (non-existent) kernel code paths, it ensures that e.g. ioctl(2) operations are always existing ioctl(2) operations.</p>
<pre><code class="lang-cpp="><span class="hljs-function"><span class="hljs-keyword">static</span> <span class="hljs-keyword">unsigned</span> <span class="hljs-keyword">long</span>
<span class="hljs-title">get_ioctl_request</span><span class="hljs-params">(<span class="hljs-keyword">void</span>)</span>
</span>{
<span class="hljs-keyword">unsigned</span> <span class="hljs-keyword">long</span> u;
<span class="hljs-comment">// ioctlprint -l -f '\t\t%x /* %n */,\n'</span>
<span class="hljs-keyword">static</span> <span class="hljs-keyword">const</span> <span class="hljs-keyword">unsigned</span> <span class="hljs-keyword">long</span> vals[] = {
<span class="hljs-number">0x8014447e</span> <span class="hljs-comment">/* DRVRESUMEDEV */</span>,
<span class="hljs-number">0xc0784802</span> <span class="hljs-comment">/* HPCFBIO_GDSPCONF */</span>,
<span class="hljs-comment">/* A lot of skipped lines here... */</span>
<span class="hljs-number">0xc0186b01</span> <span class="hljs-comment">/* KFILTER_BYNAME */</span>,
<span class="hljs-number">0x80047476</span> <span class="hljs-comment">/* TIOCSPGRP */</span>,
};
u = get_ulong() % __arraycount(vals);
<span class="hljs-keyword">return</span> vals[u];
}
</code></pre>
<p>Kamil also rewrote the HF_MEMGET() function, so instead of recharging the buffer from the fuzzer, whenever the buffer expired the fuzzed program terminates resetting its state and checks another honggfuzz input. This intends to make the fuzzing process more predictable in terms of getting good reproducers. So far we were unable to generate good reproducers and we are still working on it.</p>
<p>Unfortunately (or fortunately) the code in <a href="https://github.com/adityavardhanpadala/rumpsyscallfuzz/blob/master/honggfuzz/ioctl/ioctl_fuzz2.c">ioctl_fuzz2.c</a> triggers another (rump)kernel bug, unrelated to the reproducer in rump_panic.c. Furthermore, the crash looks like a race condition that breaks randomly, sometimes and honggfuzz doesn't generate good reproducers for it.</p>
<p>The obvious solution to this is to run the fuzzing process with TSan involved and catch such bugs quickly, unfortunately reaching MKSANITIZER for TSan + rumpkernel is still unfinished and beyond the opportunities during this GSoC.</p>
<h2 id="obstacles">Obstacles</h2>
<ul>
<li>The fuzzer is still dumb meaning that we are still using just random data as arguments to the fuzzer so the coverage did not show that much improvement b/w 13 mins and 13 hours of fuzzing.</li>
<li>Crash reproducers get sliced due to the way we are fetching input from fuzzer using HF_ITER() and as functions like <code>copyin()</code> and <code>copyout()</code> requesting quite large buffers from the fuzzers for some non-trivial functions like rump_init().</li>
</ul>
<h2 id="to-do">To-Do</h2>
<ul>
<li>Improving the crash reproduction mechanism.</li>
<li>Making the fuzzer smart by using grammar so that the arguments to syscalls are syntactically valid.</li>
<li>Finding an optimal way to fetch data from fuzzer so that the reproducers are not sliced.</li>
</ul>
<p>If the above improvements are done we get more coverage and if we are lucky enough lot more crashes.</p>
<p>Finally I'd like to thank my mentors Siddharth Muralee, Maciej Grochowski, Christos Zoulas for their guidance and Kamil Rytarowski for his constant support throughout the coding period.</p>https://blog.netbsd.org/tnf/entry/the_gnu_gdb_debugger_andThe GNU GDB Debugger and NetBSD (Part 1)Kamil Rytarowski2020-04-02T18:49:54+00:002020-04-02T18:49:54+00:00The NetBSD team of developers maintains two copies of GDB:
<ul>
<li>One in the base-system with a stack of local patches.</li>
<li>One in pkgsrc with mostly build fix patches.</li>
</ul>
<p>
The process of maintaining a modern version (GPLv3) of GDB in basesystem is
tainted with a constant extra cost. The NetBSD developers need to
rebase the stack of local patches for the newer releases of the debugger
and resurrect the support.
The GDB project is under an active development and in active refactoring
of the code, that was originally written in C, to C++.
<p>
Unfortunately we cannot abandon the local basesystem patches and rely on a pristine version
as there is lack of feature parity in the pkgsrc version of GDB: no threading support,
not operational support for most targets, no fork/vfork/etc events support,
no auxv reading support on 64-bit kernels, no proper support of signals, single step etc.
<p>
Additionally there are extra GDB patches stored in pkgsrc-wip (created by me last year), that
implement the gdbserver support for NetBSD/amd64.
gdbserver is a GDB version that makes it possible to remotely debug other programs even across different Operating Systems and CPUs.
This code has still not been merged into the mainline base-system version.
This month, I have discovered that support needs to be reworked,
as the preexisting source code directory hierarchy was rearranged.
<p>
Unless otherwise specified all the following changes were upstreamed to the mainstream GDB repository.
According to the <a href="https://sourceware.org/gdb/schedule/">GDB schedule</a>, the GDB10 branch point is planned on 2020-05-15 with release on 2020-06-05.
It's a challenge to see how much the GDB support can be improved by then for NetBSD!The NetBSD team of developers maintains two copies of GDB:
<ul>
<li>One in the base-system with a stack of local patches.</li>
<li>One in pkgsrc with mostly build fix patches.</li>
</ul>
<p>
The process of maintaining a modern version (GPLv3) of GDB in basesystem is
tainted with a constant extra cost. The NetBSD developers need to
rebase the stack of local patches for the newer releases of the debugger
and resurrect the support.
The GDB project is under an active development and in active refactoring
of the code, that was originally written in C, to C++.
<p>
Unfortunately we cannot abandon the local basesystem patches and rely on a pristine version
as there is lack of feature parity in the pkgsrc version of GDB: no threading support,
not operational support for most targets, no fork/vfork/etc events support,
no auxv reading support on 64-bit kernels, no proper support of signals, single step etc.
<p>
Additionally there are extra GDB patches stored in pkgsrc-wip (created by me last year), that
implement the gdbserver support for NetBSD/amd64.
gdbserver is a GDB version that makes it possible to remotely debug other programs even across different Operating Systems and CPUs.
This code has still not been merged into the mainline base-system version.
This month, I have discovered that support needs to be reworked,
as the preexisting source code directory hierarchy was rearranged.
<p>
Unless otherwise specified all the following changes were upstreamed to the mainstream GDB repository.
According to the <a href="https://sourceware.org/gdb/schedule/">GDB schedule</a>, the GDB10 branch point is planned on 2020-05-15 with release on 2020-06-05.
It's a challenge to see how much the GDB support can be improved by then for NetBSD!
<p>
<h1>PSIM</h1>
<p>
The GDB debugger contains <a href="https://sourceware.org/psim/">PSIM</a>
(Model of the PowerPC Architecture) originally developed by Andrew Cagney between 1994 and 1996.
This is a simulator that contains, among other things, NetBSD support in the UEA mode.
This means that GDB can run static programs prebuilt for NetBSD without execution on a real PowerPC hardware.
In order to make it work, there is need to wrap the kernel interfaces such as syscalls, errno values
and signals and handle them in the simulator.
<p>
I have updated the list of
<a href="https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=607c69321064f57595038d39af3328b0de73eb85">errno names</a>
and <a href="https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=7a20f753ef28251e4d3bca211e9ee338f67aa2a8">signal names</a>
with NetBSD 9.99.49.
<p>
It would be nice to still update the list of syscalls to reflect the current kernels,
but I have deferred this into future.
<p>
<h1>bfd changes</h1>
<p>
The AArch64 (NetBSD/evbarm) target uses PT_GETREGS and PT_GETFPREGS operation names with the same
Machine Dependent values as NetBSD/alpha and NetBSD/sparc.
This knowledge is required as these values are used in core(5) files, as emitted by a crashing program.
I've added
<a href="https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=015ec493d8603095d212df443e7b75017b572455">a patch</a>
that recognizes these ELF notes in arm64 coredumps appropriately.
<p>
I've also <a href="https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=9fcbd8a90a92bf303c41be816040789e1ed6cf4e">added</a>
a new define constant NT_NETBSDCORE_AUXV. This allows properly identifying AUXV ELF notes in core files.
Meanwhile I have implemented and
<a href="https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=06d949ec3121f4732e789db1efc53986355c9481">added</a>
detection of LWPSTATUS notes.
This note ships with
meta information (name, signal context, TLS base, etc) about threads in a process in a core.
<p>
The number of ARM and MIPS boards supported by NetBSD is huge and there are multiple variations of them.
I have <a href="https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=8b5d0a4f6ff67aa81275c705508e3f49accc8120">fixed</a>
the detection macro in bfd to recognize more arm and mips NetBSD installations.
<p>
<h1>GDB/NetBSD fixes in CPU specific files</h1>
<p>
I have reached the state of GDB being more operational for more NetBSD ports out of the box.
There were missing features and build issues that has been addressed.
I have committed the following changes:
<ul>
<li><a href="https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=25567eeece4efdea99a9a2b3a6daf81ec31b4b07">Define _KERNTYPES in vax-bsd-nat.c</a>
<li><a href="https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=52feded7781821d227db0be188e564698d29fe9e">Define _KERNTYPES in ppc-nbsd-nat.c</a>
<li><a href="https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=8110f842bc52940d27b6f1e8e5ab711238d9d210">Define _KERNTYPES in mips-nbsd-nat.c</a>
<li><a href="https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=d5be5fa4207da00d039a1d5a040ba316e7092cbd">Inherit vax_bsd_nat_target from nbsd_nat_target</a>
<li><a href="https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=12753073036aad1250b5e9a1bd6991c125150269">Add explicit cast to fix build of vax-bsd-nat.c</a>
<li><a href="https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=6227b330d563add042066259e5f933c89a85b3b5">Add support for threads in vax_bsd_nat_target</a>
<li><a href="https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=013f99f035c49b000ca058db80ad15e00aa330dc">Add support for NetBSD threads in x86-bsd-nat.c</a>
<li><a href="https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=6018d381a00515933016c539d2fdc18ad0d304b8">Inherit arm_netbsd_nat_target from nbsd_nat_target</a>
<li><a href="https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=75c56d3d1298de72aa67555f2c723a80b4818e04">Add support for NetBSD threads in arm-nbsd-nat.c</a>
<li><a href="https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=2190cf067b87e804c59c0a0c14c8ae48deabfbf4">Define _KERNTYPES in alpha-bsd-nat.c</a>
<li><a href="https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=4fed520be264b60893aa674071947890f8172915">Inherit alpha_netbsd_nat_target from nbsd_nat_target</a>
<li><a href="https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=66eaca97ebd0f1b456ffe158fce73bafcce561bb">Remove unused code from alpha-bsd-nat.c</a>
<li><a href="https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=6def66f1404e58b376655f6fb622aeb9dfc0f587">Add support for NetBSD threads in alpha-bsd-nat.c</a>
<li><a href="https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=f90280caf5b34ebd564809a7f66685efc79bbf6d">Define _KERNTYPES in m68k-bsd-nat.c</a>
<li><a href="https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=01a801176ea15ddfc988cade2e3d84c3b0abfec3">Inherit m68k_bsd_nat_target from nbsd_nat_target</a>
<li><a href="https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=bc10778499a971ab9ccecb074cc406d49e1ee608">m68k: bsd: Change type from char * to gdb_byte *</a>
<li><a href="https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=154151a6e303fa4b17e3597e3434a1c5999df8e1">Add support for NetBSD threads in m68k-bsd-nat.c</a>
<li><a href="https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=9e38d619101a7637c9aba3f722690bef127fa6a6">Include missing header to get missing declarations</a>
<li><a href="https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=9809762324491b851332ce600ae9bde8dd34f601">Inherit sh_nbsd_nat_target from nbsd_nat_target</a>
<li><a href="https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=a225c9a8692814b4a29360479aee217d73e22d50">Add support for NetBSD threads in sh-nbsd-nat.c</a>
<li><a href="https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=2108a63a5a736c2329a2a92ca58e0b9993dc5d42">Add support for NetBSD threads in sparc-nat.c</a>
<li><a href="https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=1c0aa1fbb2fb1920c12400940704ee90491b7290">Add support for NetBSD threads in amd64-bsd-nat.c</a>
<li><a href="https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=fb516a69133999df3e30fccb7c4f5759eb1090eb">Add support for NetBSD threads in i386-bsd-nat.c</a>
<li><a href="https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=c7da12c72c2eeb4f62e772df5cbb1f50f35a84ee">Add support for NetBSD threads in ppc-nbsd-nat.c</a>
<li><a href="https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=4a90f062056e842c3f53293482e0039db0da3245">Add support for NetBSD threads in hppa-nbsd-nat.c</a>
<li><a href="https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=9faa006d11a5e08264a007463435f84b77864c9c">Inherit ppc_nbsd_nat_target from nbsd_nat_target</a>
<li><a href="https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=1ff700c202465275d1ca3aee4dd979db36274294">Update the return type of gdb_ptrace to be more flexible</a>
<li><a href="https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=fcc7376e0a4c3a68ef0b9d12fcc3733416b1cc8c">Avoid get_ptrace_pid() usage on NetBSD in x86-bsd-nat.c</a>
</ul>
<p>
Now support for NetBSD in various CPU-specific files improved significantly, however there are still missing features, especially KGDB debugging and unwinding
the stack over the signal trampoline. There are still smaller or larger changes that might be needed on per-port basis and I will keep working on them.
There is need to develop at least proper aarch64 support as it is missing upstream. We might evaluate what to do with at least Itanium and RISCV.
<p>
<h1>CPU Generic improvements in the GDB codebase</h1>
<p>
I've <a href="https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=a2ecbe9fb7355698aad97841f9717cdfd7096d95">switched</a>
the <code>nbsd_nat_target::pid_to_exec_file()</code> function from a logic of reading the <code>/proc</code>
entries to a <code>sysctl(3)</code> based solution.
<p>
As the gdbserver support is around the corner, I have improved small parts of the code base to be compatibile with NetBSD.
I've <a href="https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=d3e2a5e85df4c8454135503c1034b95fecd522ab">fixed</a>
the unconditional inclusion of <code>alloca.h</code> in <code>gdbsupport</code>.
Another
<a href="https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=5a82b8a12b6a9b8167517ab1df1dcdcc4711ffda">fix</a>
namespaced a local <code>class reg</code>, because it conflicted with the <code>struct reg</code>
from the NetBSD headers.
<p>
The current logic of <code>get_ptrace_pid</code> function matches the semantics of other kernels suchs as Linux and FreeBSD.
With the guidance of upstream developers,
<a href="https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=f09db380942d1393e6f60d1ecaa7d4b6edfaaecf">I have disabled</a>
this function completely for NetBSD instead of patching it for
the NetBSD specific behavior of maintaining pairs PID+LWP for each internal <code>ptid_t</code> entry (that reflects the relation of PID, LWP and TID).
<p>
<h1>Plan for the next milestone</h1>
<p>
Finish reimplementing operational support of debugging of multi-threaded programs and upstream more patches, especially CPU-independent ones.https://blog.netbsd.org/tnf/entry/accomplishment_of_porting_ptrace_2Accomplishment of porting ptrace(2) test scenariosKamil Rytarowski2020-03-10T17:21:32+00:002020-03-10T17:21:32+00:00This month I have finished porting ptrace(2) tests from other Operating Systems.
I have determined which test scenarios were missing, compared to FreeBSD and Linux,
and integrated them into the ATF framework.
I have skipped some of the tests as the interesting behavior was already covered in
existing tests (sometimes indirectly) or tools (like picotrace), or the NetBSD kernel
exhibits different behavior.This month I have finished porting ptrace(2) tests from other Operating Systems.
I have determined which test scenarios were missing, compared to FreeBSD and Linux,
and integrated them into the ATF framework.
I have skipped some of the tests as the interesting behavior was already covered in
existing tests (sometimes indirectly) or tools (like picotrace), or the NetBSD kernel
exhibits different behavior.
<p>
As my work is reaching the end, I was trying to clean up the state with other projects.
<p>
<h1>ptrace(2) ATF tests</h1>
<p>
I have determined which test scenarios were missing and integrated them.
Certain tests like wrapping FreeBSD specific pdfork(2) call were omitted as not applicable.
<p>
There are few new tests that are marked as expected failure for corner cases that are scheduled for fixing in future.
<p>
I have also worked on SIGCHLD-based debugging and analysis of its behavior.
I have found out that <code>SA_NOCLDWAIT</code> behaves suspiciously.
This flag passed to sigaction(2) is an extension.
If set, the system will not create a zombie when the child
exits, but the child process will be automatically waited
for. The same effect can be achieved by setting the
signal handler for SIGCHLD to SIG_IGN.
Currently it behaves differently under a debugger as the child process is never collected and is waiting for parent to collect it.
According to my research this behavior is unexpected.
A potential fix might not be difficult in the kernel, but due to time constraints
I have decided to add an ATF tests for this scenario, mark it as failed and include a comment deferring this case into future.
<p>
I have also refactored the remaining threaded tests, switching them from low-level LWP API to pthread(3) one.
<p>
<h1>Other changes</h1>
<p>
I was working on
finishing projects that were left behind.
<p>
<h2>GDB and qemu upstreaming</h2>
<p>
I'm working on upstreaming NVMM support to mainline QEMU. This process is still ongoing.
<p>
I am slowly reducing the patchset against the GDB repository.
<p>
<h2>jemalloc changes</h2>
<p>
The jemalloc allocator is a general purpose malloc(3) implementation that emphasizes fragmentation avoidance and scalable concurrency support.
It's the default allocator in the NetBSD Operating System since 2007.
<p>
There are a few workarounds that make jemalloc compatible with NetBSD internals and I was trying to remove them.
Unfortunately,
the allocator tries to initialize itself too early using a C++-like constructor and intercepts the first malloc(3).
The is done before initializing libpthread, and the pthread startup code uses malloc() when registering pthread_atfork(3) callbacks.
In order to make it work, we allow premature usage of the libpthread functionality.
I was trying to correct this,
but I've introduced slight regressions in corner cases. They are hard to debug as the allocator is corrupted internally and
randomly misbehaves (hangs, occasional crashes).
I've discussed with the upstream developers about addressing this properly, but as reproducing the setup needs familiarity with the process of development NetBSD, we are still working on it.
<p>
Meanwhile, I have managed to correct known Undefined Behavior issues in jemalloc and address all known issues working together with upstream.
<p>
<h2>syzkaller</h2>
<p>
I received write access to the syzkaller GitHub repository.
I also helped to get Kernel MSan (unauthorized memory access) operational on the syzbot node.
<p>
<h2>Miscellaneous changes</h2>
<p>
I helped with the libc++ upgrade that was done by Michal Gorny (but still not merged into mainline).
As part of this work we gained a support for errno codes for POSIX robust mutexes.
<p>
I have implemented missing DT_GNU_HASH support as specified by GNU and LLVM linkers.
This code was based on the implementation from three other major BSDs.
<p>
The micro-UBSan implementation gained support for alignment_assumptions.
A number of UBSan reports were addressed.
<p>
<h1>Plan for the next and the last milestone</h1>
<p>
Upstream gdbserver support and address as many remaining bugs as the time will permit.
<p>
<h2>This work was sponsored by The NetBSD Foundation.</h2>
<p>
The NetBSD Foundation is a non-profit organization and welcomes any
donations to help us continue funding projects and services to the
open-source community. Please consider visiting the following URL to
chip in what you can:
<p>
<a href="http://netbsd.org/donations/#how-to-donate">http://netbsd.org/donations/#how-to-donate</a>https://blog.netbsd.org/tnf/entry/fundraising_2020Fundraising 2020martin2020-02-13T13:37:33+00:002020-02-14T15:42:48+00:00<p>Fundraising drive 2020: trying to raise <b>$50,000</b> for more funded development projects to fix itching issues.</p><p>Is it really more than 10 years since we last had an official fundraising drive?</p>
<p>Looking at old TNF financial reports I noticed that we have been doing quite well financially over the last years, with a steady stream of small and medium donations, and most of the time only moderate expenditures. The last fundraising drive back in 2009 was a giant success, and we have lived off it until now.</p>
<p>In the last two or three years the core team was able to find developers doing various tasks of funded development. Not all of them ended in a full success and were integrated into the main source tree — like the WiFi IEEE 802.11 rework, which still needs to be finished — but others pushed the project forward in big steps (like Support for "Arm ServerReady" compliant machines (SBBR+SBSA) debuting with the new aarch64 architecture in NetBSD 9.0).</p>
<p>There is more room for improvements, and not always volunteer time available, so funding some critical parts of development makes NetBSD better faster.</p>
<p>Besides the big development contracts we often buy hardware for developers
working on special machines, and we also invest in our server infrastructure.</p>
<p>But now it is time: we would like to officially ask for donations this year.
We are trying to raise <b>$50,000</b> in 2020, to support ongoing development and new upcoming contracts - helping to make NetBSD 10 happen this year and be the best NetBSD ever!</p>
<p>The NetBSD Foundation is a non-profit organization as per section 501(c)(3) of the Internal Revenue Code. If you are a US company or citizen, your donations may be tax deductible. Your donations may also be eligible for matching offers from your employer.</p>
<form action="https://www.paypal.com/cgi-bin/webscr" method="post">
<input type="hidden" name="cmd" value="_xclick">
<input type="hidden" name="business" value="paypal@NetBSD.org">
<input type="hidden" name="item_name" value="The NetBSD Foundation">
<input type="hidden" name="no_shipping" value="1">
<input type="hidden" name="return" value="//www.NetBSD.org/">
<input type="hidden" name="cancel_return" value="//www.NetBSD.org/">
Donate: <input type="number" NAME="amount" size="4" min="5" style="width: 4em;"> USD
<input type="image" src="https://www.netbsd.org/images/links/paypal.gif" name="submit" alt="Donate using PayPal" style="border: 0; vertical-align: middle;">
</form>https://blog.netbsd.org/tnf/entry/approaching_the_end_of_workApproaching the end of work on ptrace(2)Kamil Rytarowski2020-02-11T17:08:08+00:002020-02-11T17:08:08+00:00This is one of my last reports on enhancements on <code>ptrace(2)</code> and the surrounding code.
This month I complete a set of older pending tasks.This is one of my last reports on enhancements on <code>ptrace(2)</code> and the surrounding code.
This month I complete a set of older pending tasks.
<p>
<h1>dlinfo(3) - information about a dynamically loaded object</h1>
<p>
I've documented <code>dlinfo(3)</code> and added missing cross-reference from the <code>dlfcn(3)</code> API.
<code>dlinfo(3)</code> is a Solaris-style and first appeared in NetBSD 5.1.
Today this API is also implemented in Linux and FreeBSD, making it the right portable tool for certain operations.
Today we support only the <code>RTLD_DI_LINKMAP</code> operation out of few functionalities.
<code>RTLD_DI_LINKMAP</code> translates the <code>dlopen(3)</code> opaque handler to <code>struct link_map</code>.
<p>
This is the exact functionality needed in the LLVM sanitizers.
So far we have been using manual extraction of the struct's field with offset over an opaque pointer.
Unfortunately the existing algorithm was prone to internal modifications of the ELF loader and it used to break few times a year.
<p>
<pre>
#define _GET_LINK_MAP_BY_DLOPEN_HANDLE(handle, shift) \
((link_map *)((handle) == nullptr ? nullptr : ((char *)(handle) + (shift))))
#if defined(__x86_64__)
#define GET_LINK_MAP_BY_DLOPEN_HANDLE(handle) \
_GET_LINK_MAP_BY_DLOPEN_HANDLE(handle, 264)
#elif defined(__i386__)
#define GET_LINK_MAP_BY_DLOPEN_HANDLE(handle) \
_GET_LINK_MAP_BY_DLOPEN_HANDLE(handle, 136)
#endif
</pre>
<p>
I was pondering how to implement a dedicated interface until I found this interface in rumpkernels!
Antti Kantee, a NetBSD developer and author of rumpkernels, implemented this feature but left it undocumented.
I quickly scanned this interface, picked the man page from FreeBSD, adapted for NetBSD and pushed to LLVM.
The FreeBSD Operating system support in sanitizers also suffered from the offset struct changes and
with collaboration with me from the NetBSD side, we switched the appropriate code to a dynamic value.
This also improves compatibility with all NetBSD ports that could potentially support the LLVM sanitizers,
as previously the offsets were calculated only for x86 platforms (i386, amd64).
<p>
<h1><code>ktruss(1)</code> enhancements</h1>
<p>
I've switched <code>ktruss(1)</code> to ptrace descriptive operation names.
Previously <code>ptrace(2)</code> requests were encoded with a magic value like 0x10 or 0x14.
This style of encoding fields is not human-friendly and required checking appropriate system headers.
Now the output looks like this (reduced to ptrace syscalls only):
<pre>
3902 1 t_ptrace_waitpid ptrace(PT_ATTACH, 0xfd7, 0, 0) = 0
3902 1 t_ptrace_waitpid ptrace(PT_GET_SIGINFO, 0xfd7, 0x7f7fff51a4f0, 0x88) = 0
3902 1 t_ptrace_waitpid ptrace(PT_LWPINFO, 0xfd7, 0x7f7fff51a4c8, 0x8) = 0
3902 1 t_ptrace_waitpid ptrace(PT_LWPINFO, 0xfd7, 0x7f7fff51a4c8, 0x8) = 0
3902 1 t_ptrace_waitpid ptrace(PT_CONTINUE, 0xfd7, 0x1, 0) = 0
3902 1 t_ptrace_waitpid ptrace(PT_GET_SIGINFO, 0xfd7, 0x7f7fff51a4f0, 0x88) = 0
3902 1 t_ptrace_waitpid ptrace(PT_LWPINFO, 0xfd7, 0x7f7fff51a4c8, 0x8) = 0
3902 1 t_ptrace_waitpid ptrace(PT_LWPINFO, 0xfd7, 0x7f7fff51a4c8, 0x8) = 0
3902 1 t_ptrace_waitpid ptrace(PT_LWPINFO, 0xfd7, 0x7f7fff51a4c8, 0x8) = 0
3902 1 t_ptrace_waitpid ptrace(PT_LWPINFO, 0xfd7, 0x7f7fff51a4c8, 0x8) = 0
3902 1 t_ptrace_waitpid ptrace(PT_LWPINFO, 0xfd7, 0x7f7fff51a4c8, 0x8) = 0
3902 1 t_ptrace_waitpid ptrace(PT_CONTINUE, 0xfd7, 0x1, 0x9) = 0
5449 1 t_ptrace_waitpid ptrace(PT_TRACE_ME, 0, 0, 0) = 0
6086 1 t_ptrace_waitpid ptrace(PT_GET_SIGINFO, 0x1549, 0x7f7fffa7c230, 0x88) = 0
6086 1 t_ptrace_waitpid ptrace(PT_IO, 0x1549, 0x7f7fffa7c210, 0x20) = 0
</pre>
<p>
<h1>libc threading stubs improvements</h1>
<p>
I have adjusted the error return value of <code>pthread_sigmask(3)</code> whenever <code>libpthread</code> is either not loaded or initialized.
Now instead of returning <code>-1</code>, return <code>errno</code> on error.
This caught up after the fix in <code>libpthread</code> by Andrew Doran in 2008 in <code>lib/libpthread/pthread_misc.c</code> r.1.9.
It remains an open question whether this function shall be used without linked
in the POSIX thread library.
<p>
This incompatibility was detected by Bruno Haible (GNU) and documented in gnulib in commit
<code>pthread_sigmask: Avoid test failure on NetBSD 8.0.</code> r. 4d16a83b0c1fcb6c.
<p>
<h1>Compatibility fixes in <code>compat_netbsd32(8)</code></h1>
<p>
There was a longstanding issue with incompatibility between few syscalls in the native NetBSD ABI and 32-bit compat one.
I've addressed this with the following change:
<pre>
commit 2e2cec309dca0021e364d52597388665c500d66e
Author: kamil <kamil@NetBSD.org>
Date: Sat Jan 18 07:33:24 2020 +0000
Catch up after getpid/getgid/getuid changes in native ABI in 2008
getpid(), getuid() and getgid() used to call respectively sys_getpid(),
sys_getuid() and sys_getgid(). In the BSD4.3 compat mode there was a
fallback to call sys_getpid_with_ppid() and related functions.
In 2008 the compat ifdef was removed in sys/kern/syscalls.master r. 1.216.
For purity reasons we probably shall restore the NetBSD original behavior
and implement BSD4.3 one as a compat module, however it is not worth the
complexity.
Align the netbsd32 compat ABI to native ABI and call functions that return
two integers as in BSD4.3.
</pre>
<p>
<h1>New ptrace(2) ATF tests (OpenBSD)</h1>
<p>
I have finished importing the OpenBSD test (<code>regress/sys/ptrace/ptrace.c</code>) for unaligned program counter register.
For a long time it was cryptic what the original intention was to test,
whether it was some concept of what values should be rejected in the API for purity reasons,
whether we were testing <code>SIGILL</code>, whether there was a kernel problem or something else.
<p>
Finally, I reached the original committer (Miod Vallat) and he pointed out to me
that there was a sparc/OpenBSD bug that could break the kernel.
<p>
<a href="https://marc.info/?l=openbsd-bugs&m=107558043319084&w=2">https://marc.info/?l=openbsd-bugs&m=107558043319084&w=2</a>
<p>
Instead of hardcoding magic program counter register on a per-cpu basis, I added tests using 3 types of them for all CPUs.
At the end of the day these tests crashed the NetBSD kernel for at least a single port.
<p>
<h1>New <code>ptrace(2)</code> ATF tests (FreeBSD)</h1>
<p>
Scanning the <code>ptrace(2)</code> scenarios in FreeBSD, I decided to expand the ATF framework for
unrelated tracer variation (as opposed to real parent = debugger) for tests: <code>fork1-16</code>,
<code>vfork1-16</code>, <code>posix_spawn1-16</code>.
New tests were also introduced for:
<code>posix_spawn_detach_spawner</code>, <code>fork_detach_forker</code>, <code>vfork_detach_vforkerdone</code>,
<code>posix_spawn_kill_spawner</code>, <code>fork_kill_forker</code>, <code>vfork_kill_vforker</code>, <code>vfork_kill_vforkerdone</code>.
<p>
<h1><code>libpthread(3)</code> enhancements</h1>
<p>
I have added missing sanity checks in the <code>libpthread(3)</code> API functions.
This means that now on entry to calls like <code>pthread_mutex_lock(3)</code> we will first check whether the passed object
is a valid and alive mutex.
I have retired conditional (preprocessor variable) checks in API families (rwlock, spin-lock) as they were always enabled.
This behavior matches the Linux implementation and is assumed in LLVM sanitizers.
I also found out that these semantics are expected by third-party code, especially WolfSSL.
<p>
Our current sanity checking version of <code>pthread_equal(3)</code> found at least one abuse of the interface in the NSPR package
and its downstream users: Firefox, Thunderbird and Seamonkey.
For the time being a workaround has been applied and the bug is scheduled for proper investigation and fix.
<p>
While working on this.
I found that the jemalloc (system's malloc) initialization is misused and we initialize a jemalloc's mutex with uninitialized data.
A workaround has been applied by Christos Zoulas
but we are still looking for proper initialization model of our libc, libpthread and malloc code.
There is a problem with mutual dependencies between the components and we are looking for a clean solution.
<p>
<h1><code>env(1)</code> and <code>putenv(3)</code></h1>
<p>
The NetBSD project maintains a LLVM buildbot in the LLVM buildfarm.
We build and execute tests for virtually every LLVM project which we find important
(<code>llvm</code>, <code>clang</code>, <code>lldb</code>, <code>lld</code>, <code>compiler-rt</code>,
<code>polly</code>, <code>openmp</code>, <code>libc++</code>, <code>libc++abi</code> etc.)
A frequent issue is that LLVM developers use constructs that are portable only to a selection of OSs that are worth supporting in the tests.
One such incompatibility introduced from time to time is <code>env(1)</code> with the <code>-u</code> option.
This option is an extension to POSIX, but supported by FreeBSD, Linux and Darwin. It unsets a variable in the environment.
<p>
It was a constant cost to see this regressing over and over again on the NetBSD buildbot and
I decided to just implement it natively.
While there, I have implemented another popular option: <code>-0</code>.
This switch ends each output line with <code>NUL</code>, not newline.
<p>
While investigating the <code>env(1)</code> differences between Operating Systems,
I have learnt that <code>putenv(3)</code> changed its behavior.
This function call originally allocated its content internally, however it was changed in future versions as requested by POSIX
to not allocate memory.
I have audited the base system to see whether we obey these semantics everywhere and detected a use-after-free bug in the PAM
(Pluggable Authentication Modules framework)!
I have fixed the problem with a potential security implication.
<p>
<pre>
commit 5c19ff198d69f48222ab4907235addbfa60d4e2a
Author: kamil <kamil@NetBSD.org>
Date: Sat Feb 8 13:44:35 2020 +0000
Avoid use-after-free bug in PAM environment
Traditional BSD putenv(3) was creating an internal copy of the passed
argument. Unfortunately this was causing memory leaks and was changed by
POSIX to not allocate.
Adapt the putenv(3) usage to modern POSIX (and NetBSD) semantics.
diff --git a/usr.bin/login/login_pam.c b/usr.bin/login/login_pam.c
index 66303006b514..7d877ebeb1c0 100644
--- a/usr.bin/login/login_pam.c
+++ b/usr.bin/login/login_pam.c
@@ -1,6 +1,6 @@
-/* $NetBSD: login_pam.c,v 1.25 2015/10/29 11:31:52 shm Exp $ */
+/* $NetBSD: login_pam.c,v 1.26 2020/02/08 13:44:35 kamil Exp $ */
/*-
* Copyright (c) 1980, 1987, 1988, 1991, 1993, 1994
* The Regents of the University of California. All rights reserved.
*
@@ -37,11 +37,11 @@ __COPYRIGHT("@(#) Copyright (c) 1980, 1987, 1988, 1991, 1993, 1994\
#ifndef lint
#if 0
static char sccsid[] = "@(#)login.c 8.4 (Berkeley) 4/2/94";
#endif
-__RCSID("$NetBSD: login_pam.c,v 1.25 2015/10/29 11:31:52 shm Exp $");
+__RCSID("$NetBSD: login_pam.c,v 1.26 2020/02/08 13:44:35 kamil Exp $");
#endif /* not lint */
/*
* login [ name ]
* login -h hostname (for telnetd, etc.)
@@ -600,12 +600,12 @@ skip_auth:
*/
if ((pamenv = pam_getenvlist(pamh)) != NULL) {
char **envitem;
for (envitem = pamenv; *envitem; envitem++) {
- putenv(*envitem);
- free(*envitem);
+ if (putenv(*envitem) == -1)
+ free(*envitem);
}
free(pamenv);
}
</pre>
<p>
<h1>Other changes</h1>
<p>
I have fixed LLVM sanitizers after recent basesystem changes and they are again functional.
There was a need to adapt the code for <code>urio(4)</code> device driver removal
(USB driver for the Diamond Multimedia Rio500 MP3 player).
With the clang version 9 we were also required to adapt paths for installation of the sanitizers.
<p>
I have fixed a race in the <code>ptrace(2)</code> ATF test <code>resume1</code>.
There was a bug when two events were triggered concurrently from two competing threads.
This bug appeared after recent changes by Andrew Doran who refactored the internals and so the race was now more frequent
than before.
<p>
I am working with students and volunteers who are learning how to use sanitizers in the NetBSD context.
Over the past month we fixed <code>MKLIBCSANITIZER</code>
build with recent GCC 8.x and addressed a number of Undefined Behavior reports in the system.
<p>
I have helped to upstream chunks of the NetBSD code to GDB and GCC.
I'm working on upstreaming NVMM support to qemu. The patchset is still in review waiting for more feedback before the final merge.
<p>
I have imported <code>realpath(1)</code> utility from FreeBSD as it is used sometimes by existing scripts (even if there are more portable alternatives).
<p>
<h1>Plan for the next milestone</h1>
<p>
Port remaining <code>ptrace(2)</code> test scenarios from Linux and FreeBSD to ATF and ensure that they are properly operational.
<p>
<h2>This work was sponsored by The NetBSD Foundation.</h2>
<p>
The NetBSD Foundation is a non-profit organization and welcomes any
donations to help us continue funding projects and services to the
open-source community. Please consider visiting the following URL to
chip in what you can:
<p>
<a href="http://netbsd.org/donations/#how-to-donate">http://netbsd.org/donations/#how-to-donate</a>
https://blog.netbsd.org/tnf/entry/improving_the_ptrace_2_apiImproving the ptrace(2) API and preparing for LLVM-10.0Kamil Rytarowski2020-01-13T20:03:02+00:002020-01-13T20:03:02+00:00This month I have improved the NetBSD <code>ptrace(2)</code> API, removing one legacy
interface with a few flaws and replacing it with two new calls with new
features, and removing technical debt.
<p>
As LLVM 10.0 is branching now soon (Jan 15th 2020), I worked on proper support of
the LLVM features for NetBSD 9.0 (today RC1) and NetBSD HEAD (future 10.0).This month I have improved the NetBSD <code>ptrace(2)</code> API, removing one legacy
interface with a few flaws and replacing it with two new calls with new
features, and removing technical debt.
<p>
As LLVM 10.0 is branching now soon (Jan 15th 2020), I worked on proper support of
the LLVM features for NetBSD 9.0 (today RC1) and NetBSD HEAD (future 10.0).
<p>
<h1>ptrace(2) API changes</h1>
There are around 20 Machine Independent <code>ptrace(2)</code> calls.
The origin of some of these calls trace back to BSD4.3.
The <code>PT_LWPINFO</code> call was introduced in 2003 and was loosely inspired
by a similar interface in HP-UX <code>ttrace(2)</code>.
As that was the early in the history of POSIX threads and SMP support,
not every bit of the interface remained ideal for the current computing needs.
<p>
The <code>PT_LWPINFO</code> call was originally intended
to retrieve the thread (LWP) information inside a traced process.
<p>
This call was designed to work as an
iterator over threads to retrieve the LWP id + event information. The
event information is received in a raw format (<code>PL_EVENT_NONE</code>,
<code>PL_EVENT_SIGNAL</code>, <code>PL_EVENT_SUSPENDED</code>).
<p>
Problems:
<p>
1. <code>PT_LWPINFO</code> shares the operation name with <code>PT_LWPINFO</code> from FreeBSD
that works differently and is used for different purposes:
<p>
<ul>
<li>On FreeBSD <code>PT_LWPINFO</code> returns pieces of information for the suspended
thread, not the next thread in the iteration.
<li>FreeBSD uses a custom interface for iterating over threads (actually
retrieving the threads is done with <code>PT_GETNUMLWPS</code> + <code>PT_GETLWPLIST</code>).
<li>There is almost no overlapping correct usage of <code>PT_LWPINFO</code> on NetBSD
and <code>PL_LWPINFO</code> on FreeBSD, and this causes confusion and misuse of the
interfaces (recently I fixed such misuse in the DTrace code).
</ul>
<p>
2. pl_event can only return whether a signal was emitted to all
threads or a single one. There is no information whether this is a per-LWP
signal or per-PROC signal, no siginfo_t information is attached etc.
<p>
3. Syncing our behavior with FreeBSD would mean complete breakage of our
<code>PT_LWPINFO</code> users and it is actually unnecessary, as we receive full
<code>siginfo_t</code> through Linux-like <code>PT_GET_SIGINFO</code>, instead of reimplementing
<code>siginfo_t</code> inside ptrace_lwpinfo in FreeBSD-style. (FreeBSD
wanted to follow NetBSD and adopt some of our APIs in <code>ptrace(2)</code>
and signals.).
<p>
4. Our <code>PT_LWPINFO</code> is unable to list LWP ids in a traced
process.
<p>
5. The <code>PT_LWPINFO</code> semantics cannot be used in core files as-is (as our
<code>PT_LPWINFO</code> returns next LWP, not the indicated one) and pl_event is
redundant with <code>netbsd_elfcore_procinfo.cpi_siglwp</code>, and still
less powerful (as it cannot distinguish between a per-LWP and a per-PROC signal in a
single-threaded application).
<p>
6. <code>PT_LWPINFO</code> is already documented in the BUGS section of <code>ptrace(2)</code>,
as it contains additional flaws.
<p>
Solution:
<p>
1. Remove <code>PT_LWPINFO</code> from the public <code>ptrace(2)</code> API, keeping it only as a
hidden namespaced symbol for legacy compatibility.
<p>
2. Introduce the <code>PT_LWPSTATUS</code> that prompts the kernel about exact thread
and retrieves useful information about LWP.
<p>
3. Introduce <code>PT_LWPNEXT</code> with the iteration semantics from <code>PT_LWPINFO</code>,
namely return the next LWP.
<p>
4. Include per-LWP information in <code>core(5)</code> files as <code>"PT_LWPSTATUS@nnn"</code>.
<p>
5. Fix flattening the signal context in <code>netbsd_elfcore_procinfo</code> in
<code>core(5)</code> files, and move per-LWP signal information to the per-LWP structure
<code>"PT_LWPSTATUS@nnn"</code>.
<p>
6. Do not bother with FreeBSD like <code>PT_GETNUMLWPS</code> + <code>PT_GETLWPLIST</code> calls,
as this is a micro-optimization. We intend to retrieve the list of
threads once on attach/exec and later trace them through the LWP events
(<code>PTRACE_LWP_CREATE</code>, <code>PTRACE_LWP_EXIT</code>). It's more important to keep
compatibility with current usage of <code>PT_LWPINFO</code>.
<p>
7. Keep the existing ATF tests for <code>PT_LWPINFO</code> to avoid rot.
<p>
<code>PT_LWPSTATUS</code> and <code>PT_LWPNEXT</code> operate over newly introduced <code>"struct
ptrace_lwpstatus"</code>. This structure is inspired by:
- SmartOS <code>lwpstatus_t</code>,
- <code>struct ptrace_lwpinfo</code> from NetBSD,
- <code>struct ptrace_lwpinfo</code> from FreeBSD
<p>
and their usage in real existing open-source software.
<p>
<pre>
#define PL_LNAMELEN 20 /* extra 4 for alignment */
struct ptrace_lwpstatus {
lwpid_t pl_lwpid; /* LWP described */
sigset_t pl_sigpend; /* LWP signals pending */
sigset_t pl_sigmask; /* LWP signal mask */
char pl_name[PL_LNAMELEN]; /* LWP name, may be empty */
void *pl_private; /* LWP private data */
/* Add fields at the end */
};
</pre>
<p>
<ul>
<li><code>pt_lwpid</code> is picked from <code>PT_LWPINFO</code>.
<li><code>pl_event</code> is removed entirely as useless, misleading and harmful.
<li><code>pl_sigpend</code> and <code>pl_sigmask</code> are mainly intended to untangle the
<code>cpi_sig*</code> fields from <code>"struct ptrace_lwpstatus"</code> (fix "XXX" in the kernel
code).
<li>pl_name is an easy to use API to retrieve the LWP name, replacing
<code>sysctl()</code> retrieval. (Previous algorithm: retrieve the number of LWPs,
retrieve all LWPs; iterate over them; finding the matching ID; copy the
LWP name.) <code>pl_name</code> will also be included with the missing LWP name information
in <code>core(5)</code> files.
<li>pl_private implements currently missing interface to read the TLS
base value.
</ul>
<p>
I have decided to avoid a writable version of <code>PT_LWPSTATUS</code>
that rewrites signals, name, or private pointer. These options are
practically unused in existing open-source software. There are two
exceptions that I am familiar with, but both are specific to kludges
overusing <code>ptrace(2)</code>. If these operations are needed, they
can be implemented without a writable version of <code>PT_LWPSTATUS</code>, patching
tracee's code.
<p>
I have switched GDB (in base), LLDB, picotrace and sanitizers to the new API.
As NetBSD 9.0 is nearing release, this API change will land NetBSD 10.0
and existing <code>ptrace(2)</code> software will use <code>PT_LWPINFO</code> for now.
<p>
New interfaces are ensured to be stable and continuously verified by the ATF infrastructure.
<p>
<h1>pthreadtracer</h1>
<p>
In the early in the history of <code>libpthread</code>, the NetBSD developers
designed and programmed a <code>libpthread_dbg</code> library.
It's use-case was initially intended to handle user-space scheduling of threads
in the M:N threading model inspired by Solaris.
<p>
After the switch of the internals to new SMP design (1:1 model) by Andrew Doran,
this library lost its purpose and was no longer used
(except being linked for some time in a local base system GDB version).
I removed the <code>libpthread_dbg</code> when I modernized the <code>ptrace(2)</code> API,
as it no longer had any use
(and it was broken in several ways for years without being noticed).
<p>
As I have introduced the <code>PT_LWPSTATUS</code> call, I have decided to verify this interface
in a fancy way. I have mapped <code>ptrace_lwpstatus::pl_private</code> into the <code>tls_base</code> structure as it
is defined in the <code>sys/tls.h</code> header:
<p>
<pre>
struct tls_tcb {
#ifdef __HAVE_TLS_VARIANT_I
void **tcb_dtv;
void *tcb_pthread;
#else
void *tcb_self;
void **tcb_dtv;
void *tcb_pthread;
#endif
};
</pre>
<p>
The pl_private pointer is in fact a pointer to a structure in debugger's address space, pointing to a tls_tcl structure.
This is not true universally in every environment, but it is true in regular programs using the ELF loader and the libpthread library.
Now, with the <code>tcb_pthread</code> field we can reference a regular C-style <code>pthread_t</code> object.
Now, wrapping it into a real tracer, I have implemented a program that can either start a debuggee or attach to a process and
on demand
(as a <code>SIGINFO</code> handler, usually triggered in the BSD environment with ctrl-t)
dump the full state of <code>pthread_t</code> objects within a process. A part of the example usage is below:
<p>
<pre>
$ ./pthreadtracer -p `pgrep nslookup`
[ 21088.9252645] load: 2.83 cmd: pthreadtracer 6404 [wait parked] 0.00u 0.00s 0% 1600k
DTV=0x7f7ff7ee70c8 TCB_PTHREAD=0x7f7ff7e94000
LID=4 NAME='sock-0' TLS_TSD=0x7f7ff7eed890
pt_self = 0x7f7ff7e94000
pt_tls = 0x7f7ff7eed890
pt_magic = 0x11110001 (= PT_MAGIC=0x11110001)
pt_state = 1
pt_lock = 0x0
pt_flags = 0
pt_cancel = 0
pt_errno = 35
pt_stack = {.ss_sp = 0x7f7fef9e0000, ss_size = 4194304, ss_flags = 0}
pt_stack_allocated = YES
pt_guardsize = 65536
</pre>
<p>
Full log is stored <a href="https://netbsd.org/~kamil/ptrace/pthreadtracer.txt">here</a>.
The source code of this program, on top of picotrace is <a href="netbsd.org/~kamil/ptrace/pthreadtracer.sh">here</a>.
<p>
The problem with this utility is that it requires <code>libpthread</code> sources available and reachable by the build rules.
pthreadtracer reaches each field of <code>pthread_t</code> knowing its exact internal structure.
This is enough for validation of <code>PT_LWPSTATUS</code>,
but is it enough for shipping it to users and finding its real world use-case?
Debuggers (GDB, LLDB) using debug information can reach the same data with DWARF,
but supporting DWARF in pthreadtracer is currently harder than it ought to be for the interface tests.
There is also an option to revive at some point <code>libpthread_dbg(3)</code>, revamping it for modern <code>libpthread(3)</code>,
this would help avoid DWARF introspection and it could find some use in self-introspection programs, but are there any?
<p>
<h1>LLD</h1>
<p>
I keep searching for a solution to properly support lld (LLVM linker).
<p>
NetBSD's major issue with LLVM lld is the lack of standalone linker support,
therefore being a real GNU ld replacement. I was forced to publish a standalone wrapper for lld, called
<a href="https://github.com/netbsd/lld-standalone">lld-standalone</a>
and host it on GitHub for the time being, at least until we will sort out the talks with LLVM developers.
<p>
<h1>LLVM sanitizers</h1>
<p>
As the NetBSD code is evolving, there is a need to support multiple kernel versions starting from 9.0 with the LLVM sanitizers.
I have introduced the following changes:
<ul>
<li>[compiler-rt] [netbsd] Switch to syscall for ThreadSelfTlsTcb()</li>
<li>[compiler-rt] [netbsd] Add support for versioned statvfs interceptors</li>
<li>[compiler-rt] Sync NetBSD ioctl definitions with 9.99.26</li>
<li>[compiler-rt] [fuzzer] Include stdarg.h for va_list</li>
<li>[compiler-rt] [fuzzer] Enable LSan in libFuzzer tests on NetBSD</li>
<li>[compiler-rt] Enable SANITIZER_CAN_USE_PREINIT_ARRAY on NetBSD</li>
<li>[compiler-rt] Adapt stop-the-world for ptrace changes in NetBSD-9.99.30</li>
<li>[compiler-rt] Adapt for ptrace(2) changes in NetBSD-9.99.30</li>
</ul>
<p>
The purpose of these changes is as follows:
<ul>
<li>Stop using internal interface to retrieve the <code>tcl_tcb</code> struct (TLS base) and switch to public API with the syscall <code>_lwp_getprivate(2)</code>.
While there, I have harmonized the namespacing of <code>__lwp_getprivate_fast()</code> and <code>__lwp_gettcb_fast()</code> in the NetBSD distribution.
Now, every port will need to use the same define
(<code>-D_RTLD_SOURCE</code>, <code>-D_LIBC_SOURCE</code> or <code>-D__LIBPTHREAD_SOURCE__</code>).
Previously these interfaces were conflicting with the public namespaces (affecting kernel builds) and wrongly suggesting that these
interfaces might be available to public third party code. Initially I used it in LLVM sanitizers, but switched it to full-syscall
<code>_lwp_getspecific()</code>.</li>
<li>Nowadays almost every mainstream OS implements support for preinit/initarray/finitarray in all ports, regardless of ABI requirements.
NetBSD originally supported these features only when they were mandated by an ABI specification.
Christos Zoulas in 2018 enabled these features for all CPUs, and this eventually allowed to enable this
feature unconditionally for consumption in the sanitizer code. This allows use of the same interface as Linux or Solaris,
rather than relying on C++-style constructors that have their own issues (need to abuse priorities of constructors and lack of guarantee
that our code will be called before other constructors, which can be fatal).</li>
<li>Support for kernels between 9.0 and 9.99.30 (and later, unless there are breaking changes).</li>
</ul>
<p>
There is still one portability issue in the sanitizers, as we hard-code the offset of the <code>link_map</code> field within the internal <code>dlopen</code> handle pointer.
The <code>dlopen</code> handler is internal to the ELF loader object of type <code>Obj_Entry</code>.
This type is not available to third party code and it is not stable.
It also has a different layout depending on the CPU architecture.
The same problem exists for at least FreeBSD, and to some extent to Linux.
I have prepared a <a href="http://netbsd.org/~kamil/patch-00213-compiler-rt-dlinfo.txt">patch</a> that utilizes
the <code>dlinfo</code>(3) call with option <code>RTLD_DI_LINKMAP</code>.
Unfortunately there is a regression with MSan on NetBSD HEAD (it works on 9.0rc1) that makes it harder for me to finalize the patch.
I suspect that after the switch to GCC 8,
there is now incompatible behavior that causes a recursive call
sequence: <code>_Unwind_Backtrace()</code> calling
<code>_Unwind_Find_FDE()</code>, calling <code>search_object</code>, and triggering the
<code>__interceptor_malloc</code> interceptor again, which calls <code>_Unwind_Backtrace()</code>, resulting in deadlock.
The offending code is located in <code>src/external/gpl3/gcc/dist/libgcc/unwind-dw2-fde.c</code> and needs proper investigation.
A quick workaround to stop recursive stack unwinding unfortunately did not work, as
there is another (related?) problem:
<p>
<pre>
==4629==MemorySanitizer CHECK failed:
/public/llvm-project/llvm/projects/compiler-rt/lib/msan/msan_origin.h:104 "((stack_id)) != (0)" (0x0, 0x0)
</pre>
<p>
This shows that this low-level code is very sensitive to slight changes, and needs maintenance power.
We keep improving the coverage of tested scenarios on the LLVM buildbot, and we enabled sanitizer tests on 9.0 NetBSD/amd64;
however we could make use of more manpower in order to reach full Linux parity in the toolchain.
<p>
<h1>Other changes</h1>
<p>
As my project in LLVM and ptrace(2) is slowly concluding,
I'm trying to finalize the related tasks that were left behind.
<p>
I've finished researching why we couldn't use syscall restart on
<a href="https://reviews.llvm.org/D58230">kevent(2) call in LLDB</a> and improved the
system documentation on it.
I have also fixed small nits in the NetBSD
<a href="https://wiki.netbsd.org/tutorials/kqueue_tutorial/">wiki page</a> on <code>kevent(2)</code>.
<p>
I have updated the list of ELF defines
for CPUs and OS ABIs in <code>sys/exec_elf.h</code>.
<p>
<h1>Plan for the next milestone</h1>
<p>
Port remaining ptrace(2) test scenarios from Linux, FreeBSD and OpenBSD to ATF and ensure that they are properly operational.
<p>
<h2>This work was sponsored by The NetBSD Foundation.</h2>
<p>
The NetBSD Foundation is a non-profit organization and welcomes any
donations to help us continue funding projects and services to the
open-source community. Please consider visiting the following URL to
chip in what you can:
<p>
<a href="http://netbsd.org/donations/#how-to-donate">http://netbsd.org/donations/#how-to-donate</a>
https://blog.netbsd.org/tnf/entry/board_of_directors_and_officersBoard of Directors and Officers electedWilliam J. Coldwell2019-11-20T21:12:59+00:002019-11-21T10:06:29+00:00<p>
Per the membership voting, we have seated the new Board of Directors of the NetBSD Foundation:
</p>
<ul>
<li>Taylor R. Campbell <riastadh@></li>
<li>William J. Coldwell <billc@></li>
<li>Michael van Elst <mlelstv@></li>
<li>Thomas Klausner <wiz@></li>
<li>Cherry G. Mathew <cherry@></li>
<li>Pierre Pronchery <khorben@></li>
<li>Leonardo Taccari <leot@></li>
</ul>
<p>
We would like to thank Makoto Fujiwara <mef@> and Jeremy C. Reed <reed@> for their service on the Board of Directors during their term(s).
</p>
<p>
The new Board of Directors have voted in the executive officers for The NetBSD Foundation:
</p>
<table>
<tr><td>President:</td><td>William J. Coldwell</td></tr>
<tr><td>Vice President:</td><td> Pierre Pronchery</td></tr>
<tr><td>Secretary: </td><td>Christos Zoulas</td></tr>
<tr><td>Assistant Secretary:</td><td> Thomas Klausner</td></tr>
<tr><td>Treasurer:</td><td> Christos Zoulas</td></tr>
<tr><td>Assistant Treasurer:</td><td> Taylor R. Campbell</td></tr>
</table>
<p>
Thanks to everyone that voted and we look forward to a great 2020.
</p>
https://blog.netbsd.org/tnf/entry/stabilization_of_the_ptrace_21Stabilization of the ptrace(2) threads continuedKamil Rytarowski2019-11-04T10:57:53+00:002019-11-04T11:05:04+00:00I have introduced changes to make debuggers more reliable in threaded scenarios.
Additionally I have revamped micro-UBSan runtime for newer Clang (version 10git).
I have received the OK from core@ to switch our iconv(3) to POSIX conformant iconv(3)
and I have adapted where possible and readily known in pkgsrc to the newer API.
This month I continued to find a solution to the impasse in LLD that blocks adding NetBSD support.I have introduced changes to make debuggers more reliable in threaded scenarios.
Additionally I have revamped micro-UBSan runtime for newer Clang (version 10git).
I have received the OK from core@ to switch our iconv(3) to POSIX conformant iconv(3)
and I have adapted where possible and readily known in pkgsrc to the newer API.
This month I continued to find a solution to the impasse in LLD that blocks adding NetBSD support.
<p>
<h1>Threading support</h1>
<p>
I have simplified the struct proc and removed a <code>p_oppid</code> field that stored the numeric process id of the original parent (forker).
This field is not needed as it duplicates <code>p_opptr</code> (current real parent pointer) that is already safe to
use. So far this has not proven to be unsafe.
<p>
I have refactored the signal code making it more verbose to reflect the actual needs of the kernel signal code.
<p>
I have fixed a nasty bug in the function that is called when a thread returns from the kernel to userland.
There was a tiny time window when in certain scenarios a thread was never stopped on process suspension
but was instead resumed causing waitpid(2) polling to never return success as the process can be never stopped with
a running thread.
<p>
There was a race bug that could cause a nested thread termination call, triggering a panic.
<p>
With the above changes I was able to reliably run all ATF tests for LWP events (threading events).
I have also bumped the threading tests to atually execute 100 concurrent threads,
as the higher number can more easily trigger anomalies.
In my observations all tests are now rock solid.
<p>
There are now no longer any ptrace(2) tests in ATF marked as flaky or disabled.
The two main offenders, vfork(2) events and threading events, are now solid.
<p>
Michal Gorny detected another source of instability of threads with a LLDB regression test.
It was related to emitting a massive number of concurrent threads.
I have helped Michal to address this problem and squash the bug.
<p>
All of the above changes are now pulled to NetBSD-9 for future 9.0 release.
<p>
There are at the time of writing, 4 failing LLDB threading tests and few more related to debug registers.
Both failure types are under investigation.
They could be bugs in the NetBSD support in some extent, but maybe there is need to fixup something on the kernel level.
<p>
The project is still not 100% accomplished but we are now very close to finishing everything in the domain of threads.
I could torture the NetBSD kernel for few hours with a massive number of threads and events without a single crash or failure.
On the other hand there are still likely some suspicious corner cases that need proper investigation.
There are also some suspicious reports for crashes from syzkaller, the kernel fuzzer.
Those still need to be promptly checked.
<p>
<h1>LLVM projects</h1>
<p>
I have attempted to change our original plan with LLD and instead of mutating the LLD behavior on target basis,
write a dedicated LLD wrapper that tunes LLD for NetBSD.
My patch is still in review. As an improvement over the previous ones, it wasn't immediately rejected...
<a href="https://reviews.llvm.org/D69755">https://reviews.llvm.org/D69755</a>.
<p>
I have upstreamed chunks of code with the following commits:
<ul>
<li>[compiler-rt] [msan] Correct the __libc_thr_keycreate prototype
<li>[compiler-rt] [msan] Support POSIX iconv(3) on NetBSD 9.99.17+
<li>[compiler-rt] Harmonize __sanitizer_addrinfo with the NetBSD headers
<li>[compiler-rt] Sync NetBSD syscall hooks with 9.99.17
</ul>
<p>
<h1>NetBSD distribution changes</h1>
<p>
I have switched the iconv(3) function prototype to POSIX-conformant form.
The history of this function is documented in iconv(3) as follows:
<p>
<pre>
STANDARDS
iconv_open(), iconv_close(), and iconv() conform to IEEE Std 1003.1-2001
("POSIX.1").
Historically, the definition of iconv has not been consistent across
operating systems. This is due to an unfortunate historical mistake,
documented in this e-mail:
https://www5.opengroup.org/sophocles2/show_mail.tpl?&source=L&listname=austin-group-l&id=7404.
The standards page for the header file <iconv.h> defined the second
argument of iconv() as char **, but the standards page for the iconv()
implementation defined it as const char **. The standards committee
later chose to change the function definition to follow the header file
definition (without const), even though the version with const is
arguably more correct. NetBSD used initially the const form. It was
decided to reject the committee's regression and become (technically)
incompatible.
This decision was changed in NetBSD 10 and the iconv() prototype was
synchronized with the standard.
</pre>
<p>
Meanwhile I fixed what was known to be effected in pkgsrc.
Unfortunately Qt4/KDE4 had several build issues and this motivated me to fix its users
for the new function through upgrades to the Qt5/KDE5 stack.
Many dead packages without upgrade path were dropped from pkgsrc.
<p>
As there is a new Clang upgrade coming, I have implemented handlers for new
UBSan reports: function_type_mismatch_v1() and implicit_conversion().
The first one is a new ABI for function_type_mismatch() and the second one is
completely new.
<p>
<h1>GSoC Mentor Summit</h1>
<p>
I took part in the GSoC Mentor Summit in Munich and presented a talk titled
<a href="//netbsd.org/~kamil/GSoC2019.html">"NetBSD version 9. What's new in store?"</a>.
<p>
<h1>Plan for the next milestone</h1>
<p>
Support Michal Gorny in reaching the milestone of passing all threading and debug register tests in LLDB.
<p>
<h2>This work was sponsored by The NetBSD Foundation.</h2>
<p>
The NetBSD Foundation is a non-profit organization and welcomes any
donations to help us continue funding projects and services to the
open-source community. Please consider visiting the following URL to
chip in what you can:
<p>
<a href="http://netbsd.org/donations/#how-to-donate">http://netbsd.org/donations/#how-to-donate</a>https://blog.netbsd.org/tnf/entry/stabilization_of_the_ptrace_2Stabilization of the ptrace(2) threadsKamil Rytarowski2019-10-10T10:24:10+00:002019-10-10T10:32:38+00:00I have introduced changes that make debuggers more reliable in threaded scenarios.
Additionally, I have enhanced Leak Sanitizer support and
introduced various improvements in the basesystem.I have introduced changes that make debuggers more reliable in threaded scenarios.
Additionally, I have enhanced Leak Sanitizer support and
introduced various improvements in the basesystem.
<p>
<h1>Threading support</h1>
<p>
Threads and synchronization in the kernel, in general, is an evergreen task of the kernel developers.
The process of enhancing support for tracing multiple threads has been documented by Michal Gorny in
his LLDB entry
<a href="//blog.netbsd.org/tnf/entry/threading_support_in_lldb_continued">Threading support in LLDB continued</a>.
<p>
Overall I have introduced these changes:
<ul>
<li>Separate suspend from userland (_lwp_suspend(2)) flag from suspend by a debugger (PT_SUSPEND).
This removes one of the underlying problems of threading stability as a debuggee was able to accidentally
unstop suspended thread. This property is needed whenever we want to trace a selection (typically single entity) of threads.</li>
<li>Store SIGTRAP event information inside siginfo_t, rather than in struct proc.
A single signal can only be reported at the time to the debugger, and its context is no longer prone to be overwritten by concurrent threads.</li>
<li>Change that introduces restarts in functions notifying events for debuggers. There was a time window between registering an event by a thread, stopping the process
and unlocking mutexes of the process; as another process could take the mutexes before being stopped and overwrite the event with its own data.
Now each event routine for debugger checks whether a process is already stopping (or demising or no longer being tracked) and preserves the signal to be emitted
locally in the context of the lwp local variable on the stack and continues stopping self as requested by the other LWP. Once the thread is awaken, it retries to emit
the signal and deliver the event signal to the debugger.</li>
<li>Introduce PT_STOP, that combines kill(SIGSTOP) and ptrace(PT_CONTINUE,SIGSTOP) semantics in a single call.
It works like:
<ul>
<li>kill(SIGSTOP) for unstopped tracee</li>
<li>ptrace(PT_CONTINUE,SIGSTOP) for stopped tracee</li>
</ul>
The child will be stopped and always possible to be waited (with wait(2)
like calls).
<p>
For stopped tracee kill(SIGSTOP) has no effect. PT_CONTINUE+SIGSTOP cannot
be used on an unstopped process (EBUSY).
<p>
This operation is modeled after PT_KILL that is similar for the SIGKILL
call. While there, allow PT_KILL on unstopped traced child.
<p>
This operation is useful in an abnormal exit of a debugger from a signal
handler, usually followed by waitpid(2) and ptrace(PT_DETACH).</li>
</ul>
<p>
For the sake of tracking the missed in action signals emitted by tracee,
I have introduced the feature in NetBSD truss (as part of the
<a href="https://github.com/krytarowski/picotrace">picotrace</a>
repository) to register syscall entry (SCE) and syscall exit (SCX) calls and
track missing SCE/SCX events that were never delivered.
Unfortunately, the number of missing events was huge, even for simple 2-threaded applications.
<p>
<pre>
truss[2585] running for 22.205305922 seconds
truss[2585] attached to child=759 ('firefox') for 22.204289369 seconds
syscall seconds calls errors missed-sce missed-scx
read 0.048522952 609 0 54 76
write 0.044693735 487 0 35 66
open 0.002516815 18 0 5 5
close 0.001015263 17 0 9 6
unlink 0.001375463 13 0 3 0
getpid 0.093458089 1993 0 16 56
geteuid 0.000049301 1 0 0 1
recvmsg 0.343353019 4828 3685 90 112
access 0.001450653 12 3 5 4
dup 0.000570904 10 0 0 1
munmap 0.010375949 88 0 6 3
mprotect 0.196781932 2251 0 11 62
madvise 0.049820002 430 0 11 18
writev 0.237488362 1507 0 76 67
rename 0.000379918 2 0 1 0
mkdir 0.000283846 2 2 1 2
mmap 0.033342935 481 0 15 40
lseek 0.003341775 62 0 25 24
ftruncate 0.000507707 9 0 1 0
__sysctl 0.000144506 2 0 0 0
poll 18.694195617 4531 0 106 191
__sigprocmask14 0.001585329 20 0 0 2
getcontext 0.000083238 1 0 0 0
_lwp_create 0.000104646 1 0 0 0
_lwp_self 0.001456718 22 0 24 79
_lwp_unpark 0.035319633 607 0 14 39
_lwp_unpark_all 0.020660377 250 0 38 50
_lwp_setname 0.000118418 2 0 0 0
__select50 15.125525493 637 0 82 125
__gettimeofday50 3.279021049 2930 0 40 135
__clock_gettime50 10.673311747 33132 0 1418 3003
__stat50 0.006375356 52 3 12 5
__fstat50 0.001490944 17 0 3 2
__lstat50 0.000110906 1 0 1 0
__getrusage50 0.008863815 109 0 7 1
___lwp_park60 62.720893458 964 251 454 453
------------- ------- ------- ------- -------
111.638589870 56098 3944 2563 4628
</pre>
<p>
With my kernel changes landed, the number of missed sce/scx events is down to <b>zero</b>
(with exceptions to signals that e.g. never return such as the exit(2) call).
<p>
Once these changes settle in HEAD, I plan to backport them to NetBSD-9.
I have already received <a href="//gnats.netbsd.org/54541">feedback</a> that GDB works much better now.
<p>
The kernel also has now more runtime asserts that validate correctness of the code paths.
<p>
<h1>Sanitizers</h1>
<p>
I've introduced a special preprocessor macro to detect LSan (__SANITIZE_LEAK__) and UBSan (__SANITIZE_UNDEFINED__)
in GCC.
The patches were submitted upstream to the GCC mailing list, in two patches
(<a href="https://gcc.gnu.org/ml/gcc-patches/2019-09/msg01270.html">LSan</a> +
<a href="https://gcc.gnu.org/ml/gcc-patches/2019-09/msg01286.html">UBSan</a>).
Unfortunately, GCC does not see value in feature parity with LLVM and for the time being it will be a local NetBSD specific GCC extension.
These macros are now integrated into the NetBSD public system headers, for use by the basesystem software.
<p>
The LSan macro is now used inside the LLVM codebase and the ps(1) program is the first user of it.
The UBSan macro is now used to disable relaxed alignment on x86.
While such code is still functional, it is not clean from undefined behavior as specified by C.
This is especially needed in the kernel fuzzing process, as we can reduce noise from less interesting reports.
<p>
During the previous month a number of reports from kernel fuzzing were fixed.
There is still more to go.
<p>
Almost all local patches needed for LSan were merged upstream.
The last remaining local patch is scheduled for later as it is very invasive for all platforms and sanitizers.
In the worst case we just have more false negatives in detection of leaks in specific scenarios.
<p>
<h1>Miscellaneous changes</h1>
<p>
I have fixed a regression in upstream GDB with SIGTTOU handling.
This was an upstream bug fixed by Alan Hayward and cherry-picked by me.
As a side effect, a certain environment setup would cause the tracer to sleep.
<p>
I have reverted the regression in changed in6_addr change. It appeased UBSan, but broke at least qemu networking.
The regression was tracked down by Andreas Gustafsson and reported in the NetBSD's bug tracking system.
<p>
I have landed a patch that returns ELF loader dl_phdr_info information for dl_iterate_phdr(3).
This synchronized the behavior with Linux, FreeBSD and OpenBSD and is used by sanitizers.
<p>
I have passed through core@ the patch to change the kevent::udata type from intptr_t to void*.
The former is slightly more pedantic, but the latter is what is in all other kevent users and this mismatch of types affected
specifically C++ users that needed special NetBSD-only workarounds.
<p>
I have marked git and hg meta files as ones to be ignored by cvs import.
This was causing problems among people repackaging the NetBSD source code with other VCS software than CVS.
<p>
I keep working on getting GDB test-suite to run on NetBSD, I spent some time on getting fluent in the TCL programming language
(as GDB uses dejagnu and TCL scripting).
I have already fixed two bugs that affected NetBSD users in the TCL runtime: getaddrbyname_r and gethostbyaddr_r were falsely reported as
available and picked on NetBSD, causing damage in operation.
Fluency in TCL will allow me to be more efficient in addressing and debugging failing tests in GDB and likely reuse this
knowledge in other fields useful for the project.
<p>
I made __CTASSERT a static assert again.
Previously, this homegrown check for compile-time checks silently stopped working for C99 compilers supporting VLA (variable length array).
It was caught by kUBSan that detected VLA of dynamic size of -1, that is still compatible but has unspecified runtime semantics.
The new form is inspired by the Perl ctassert code and uses bit-field constant that enforces the assert to be effective again.
Few misuses __CTASSERT, mostly in the Linux DRMKMS code, were fixed.
<p>
I have submitted a proposal to the C Working Group a proposal to add new <a href="http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2419.htm">methods for setting and getting the thread name</a>.
<p>
<h1>Plan for the next milestone</h1>
<p>
Keep stabilizing the reliability debugging interfaces and get ATF and LLDB threading code reliably pass tests.
Cover more scenarios with ptrace(2) in the ATF regression test-suite.
<p>
<h2>This work was sponsored by The NetBSD Foundation.</h2>
<p>
The NetBSD Foundation is a non-profit organization and welcomes any
donations to help us continue funding projects and services to the
open-source community. Please consider visiting the following URL to
chip in what you can:
<p>
<a href="http://netbsd.org/donations/#how-to-donate">http://netbsd.org/donations/#how-to-donate</a>
https://blog.netbsd.org/tnf/entry/eurobsd_2019EuroBSDCon 2019Kamil Rytarowski2019-09-25T08:40:08+00:002019-09-25T10:18:06+00:00Submitted by Maciej Grochowski.
<p>
This year This year <a href="https://2019.eurobsdcon.org/">EuroBSDCon</a> took place in Lillehammer Norway. I had the pleasure to
attend as a speaker with my talk about fuzzing the NetBSD filesystems.Submitted by Maciej Grochowski.
<p>
This year <a href="https://2019.eurobsdcon.org/">EuroBSDCon</a> took place in Lillehammer Norway. I had the pleasure to
attend as a speaker with my talk about fuzzing the NetBSD filesystems.
<p>
<h1>Venue</h1>
<p>
Lillehammer is a ski resort, nestled amid very beautiful scenery between
mountains and lakes, just two hours from Oslo. The conference took place
in the Scandic Lillehammer Hotel, a little bit away from the downtown of
Lillehammer, close to the Olympic Ski Jumps.
<p>
<center><img src="//netbsd.org/~kamil/eurobsd2019/skijump.jpeg"></center>
<center><b>View from the Olympic Ski Jump</b></center>
<p>
<h1>Talks</h1>
<p>
Every year, EuroBSDCon has a lot of interesting talks. Unfortunately, it is
hard to attend all the interesting seminars, as many of them take place at
the same time, so I won't be able to highlight all of them; accordingly, I
gratefully acknowledge several organizations for handling the live
streaming from every session.
<p>
<h1>Keynote: Embedded Ethics</h1>
<p>
The conference started with an excellent Keynote from Patricia Aas
(ex. Opera/Cisco/Vivaldi, cur Turtlesec), about the Ethics in the IT
industry. As a person who is familiar with the issues with the privacy and
many different threads of abusing user data by the company, I have to say
that this talk started the avalanche of different thoughts and reflections
in my mind. To my surprise, I was not the only one to have such thoughts.
This topic arose quite often during the rest of the conference through many
conversations between different people. For those of you who didn't see it
yet, I highly recommend that you do. The key takeaway is that we, the people
who are building today's digital world, need to think about the implications
of our work and decisions upon the users of our services. This topic
is getting more complicated even as we think about it. However, Patricia
come here with the strategy "Annoying as a Service" that can be simply
used in every situation to at least not makes things worse...
<p>
<h1>Conference Talks</h1>
<p>
During the first day, there were a couple of interesting talks about
NetBSD: "Improving modularity of NetBSD compat code", and mine, on
"<a href="//www.netbsd.org/~kamil/Maciej_Grochowski-FS_Fuzzing_EuroBSDCon2019.pdf">Fuzzing NetBSD Filesystems"</a> [+ <a href="//netbsd.org/~kamil/eurobsdcon2019_fuzzing/presentation.html">
Taking NetBSD kernel bug roast to the next level: Kernel Fuzzers (quick A.D. 2019 overview)
</a> by Kamil Rytarowski]. As it turns out,
there was another interesting talk about foundations of kernel fuzzing by
Andrew Turner, in which he presented the connection between sanitizers,
tracing modes and fuzzers. After the break, I attended the excellent talk
"7th Edition Unix at 40" by Warner Losh -- if you love the history of
Unix, this is a must-see. The first day finished with the social mixer.
The second day started with one of my favourites of the entire conference:
"Kernel TLS and TLS hardware offload" via Drew Gallatin and Hans Petter
Selasky. In another room was also a very interesting seminar on Rust for
System Programmers. The next session via Netflix folks was about NUMA
optimizations in the FreeBSD Network stack, another interesting
talk about the usage of BSD as a high-speed CDN serving about 200Gbps
Video content(!). After that, I attended the session on The Future of
OpenZFS via Allan Jude, where he showed the progress done in the
collaboration of different OSes on ZFS Filesystem. The last sessions I
attended were the "23 years of software side-channel attacks" by Colin,
and the last one before the closing notes: "Unbound & FreeBSD: A true love
story", by Pablo Carboni.
<p>
<h1>Highlights</h1>
<p>
<ul>
<li>
Security: We can see clearly that the BSD community continues efforts
for making BSDs more secure on various levels. This year we talked
mostly about fuzzing, and in this area, it is impossible not to
recognise NetBSD for great progress.
<li>
CDN use-case: Netflix contributions to FreeBSD make it a great system
for CDN, year after year innovating and increasing the performance. I
hope we will see more companies using BSDs as core for their CDN
infrastructure.
<li>
ZFS: The filesystem has come a long way, despite being a project divided
between different communities. Now thanks to the efforts of the
developers, OpenZFS as a united community will be able to progress
even faster and take advantage of projects that are using it. I
believe the OpenZFS initiative is one of the most important steps taken
by the community in many years.
</ul>
<p>
<h1>Social Event</h1>
<p>
This year's social event took place in the Open Air Museum in Maihaugen,
where we were able to see, preserved in excellent condition, parts of the
Norwegian houses from the 19th century through the late 20th century. The
fun part was that every house was open and you were able to go inside,
some of them with people dressed up in the fashion of the same years,
talking about the age. I very much enjoyed it, as it was a great
opportunity to learn more about Norwegian culture and history.
<p>
<center><img src="//netbsd.org/~kamil/eurobsd2019/city.jpg"></center>
<center><b>The XX century city</b></center>
<p>
<center><img src="//netbsd.org/~kamil/eurobsd2019/school.jpg"></center>
<center><b>XIX century school</b></center>
<p>
<h1>Next Year!</h1>
<p>
The most important key point during closing notes is always: "where will the
next EuroBSDCon take place?!" This year the guessing game was:
<p>
<ul>
<li> Beer will be cheaper than in Norway
<li> [picture of Schnitzel]
<li> Photo of...
</ul>
<p>
Vienna!
<p>
<center><img src="//netbsd.org/~kamil/eurobsd2019/vienna.jpg"></center>
<p>
<b>Hope to see you all next year in Vienna!</b>
<p>https://blog.netbsd.org/tnf/entry/llvm_santizers_and_gdb_regressionLLVM santizers and GDB regression test suiteKamil Rytarowski2019-09-03T15:00:09+00:002019-09-03T15:08:33+00:00As NetBSD-9 is branched, I have been asked to finish the LLVM sanitizer integration.
This work is now accomplished and with MKLLVM=yes build option (by default off),
the distribution will be populated with LLVM files for
ASan, TSan, MSan, UBSan, libFuzzer, SafeStack and XRay.
<p>
I have also transplanted basesystem GDB patched to my GDB repository and managed to run the GDB regression test-suite.As NetBSD-9 is branched, I have been asked to finish the LLVM sanitizer integration.
This work is now accomplished and with MKLLVM=yes build option (by default off),
the distribution will be populated with LLVM files for
ASan, TSan, MSan, UBSan, libFuzzer, SafeStack and XRay.
<p>
I have also transplanted basesystem GDB patched to my GDB repository and managed to run the GDB regression test-suite.
<p>
<h1>NetBSD distribution changes</h1>
<p>
I have enhanced and imported my local MKSANITIZER code that makes whole distribution sanitization possible.
Few real bugs were fixed and a number of patches were newly written to reflect the current NetBSD sources state.
I have also merged another chunk of the fruits of the GSoC-2018 project with fuzzing the userland (by plusun@).
<p>
The following changes were committed to the sources:
<ul>
<li>ab7de18d0283 Cherry-pick upstream compiler-rt patches for LLVM sanitizers</li>
<li>966c62a34e30 Add LLVM sanitizers in the MKLLVM=yes build</li>
<li>8367b667adb9 telnetd: Stop defining the same variables concurrently in bss and data</li>
<li>fe72740f64bf fsck: Stop defining the same variable concurrently in bss and data</li>
<li>40e89e890d66 Fix build of t_ubsan/t_ubsanxx under MKSANITIZER</li>
<li>b71326fd7b67 Avoid symbol clashes in tests/usr.bin/id under MKSANITIZER</li>
<li>c581f2e39fa5 Avoid symbol clashes in fs/nfs/nfsservice under MKSANITIZER</li>
<li>030a4686a3c6 Avoid symbol clashes in bin/df under MKSANITIZER</li>
<li>fd9679f6e8b1 Avoid symbol clashes in usr.sbin/ypserv/ypserv under MKSANITIZER</li>
<li>5df2d7939ce3 Stop defining _rpcsvcdirty in bss and data</li>
<li>5fafbe8b8f64 Add missing extern declaration of ib_mach_emips in installboot</li>
<li>d134584be69a Add SANITIZER_RENAME_CLASSES in bsd.prog.mk</li>
<li>2d00d9b08eae Adapt tests/kernel/t_subr_prf for MKSANITIZER</li>
<li>ce54363fe452 Ship with sanitizer/lsan_interface.h for GCC 7</li>
<li>7bd5ee95e9a0 Ship with sanitizer/lsan_interface.h for LLVM 7</li>
<li>d8671fba7a78 Set NODEBUG for LLVM sanitizers</li>
<li>242cd44890a2 Add PAXCTL_FLAG rules for MKSANITIZER</li>
<li>5e80ab99d9ce Avoid symbol clashes in test/rump/modautoload/t_modautoload with sanitizers</li>
<li>e7ce7ecd9c2a sysctl: Add indirection of symbols to remove clash with sanitizers</li>
<li>231aea846aba traceroute: Add indirection of symbol to remove clash with sanitizers</li>
<li>8d85053f487c sockstat: Add indirection of symbols to remove clash with sanitizers</li>
<li>81b333ab151a netstat: Add indirection of symbols to remove clash with sanitizers</li>
<li>a472baefefe8 Correct the memset(3)'s third argument in i386 biosdisk.c</li>
<li>7e4e92115bc3 Add ATF c and c++ tests for TSan, MSan, libFuzzer</li>
<li>921ddc9bc97c Set NOSANITIZER in i386 ramdisk image</li>
<li>64361771c78d Enhance MKSANITIZER support</li>
<li>3b5608f80a2b Define target_not_supported_body() in TSan, MSan and libFuzzer tests</li>
<li>c27f4619d513 Avoids signedness bit shift in db_get_value()</li>
<li>680c5b3cc24f Fix LLVM sanitizer build by GCC (HAVE_LLVM=no)</li>
<li>4ecfbbba2f2a Rework the LLVM compiler_rt build rules</li>
<li>748813da5547 Correct the build rules of LLVM sanitizers</li>
<li>20e223156dee Enhance the support of LLVM sanitizers</li>
<li>0bb38eb2f20d Register syms.extra in LLVM sanitizer .syms files</li>
</ul>
<p>
Almost all of the mentioned commits were backported to NetBSD-9 and will land 9.0.
<p>
As a demo, I have crafted a writing on combining RUMPKERNEL, MKSANITIZER with the honggfuzz fuzzer:
<a href="//netbsd.org/~kamil/rump/rump_pub_etfs_register_buffer.c">Rumpkernel assisted fuzzing of the NetBSD file system kernel code in userland</a>.
<p>
<h1>GDB</h1>
I've merged NetBSD distribution downstream GDB patches into my local GDB tree and executed the regression tests (check-gdb):
<p>
<pre>
[...]
Test run by kamil on Mon Sep 2 12:36:03 2019
Native configuration is x86_64-unknown-netbsd9.99
=== gdb tests ===
Schedule of variations:
unix
[...]
=== gdb Summary ===
# of expected passes 54591
# of unexpected failures 3267
# of expected failures 35
# of unknown successes 3
# of known failures 59
# of unresolved testcases 29
# of untested testcases 141
# of unsupported tests 399
</pre>
<p>
Full log is
<a href="http://netbsd.org/~kamil/gdb/test-gdb.patched-8.3.50.20190716-git.gdb.sum">here</a>.
<p>
This means that there are a lot of more tests and known failures than in 2017-09-05:
<pre>
$ uname -a
NetBSD chieftec 8.99.2 NetBSD 8.99.2 (GENERIC) #0: Sat Sep 2 22:55:29 CEST 2017 root@chieftec:/public/netbsd-root/sys/arch/amd64/compile/GENERIC amd64
Test run by kamil on Tue Sep 5 17:06:28 2017
Native configuration is x86_64--netbsd
=== gdb tests ===
Schedule of variations:
unix
[...]
=== gdb Summary ===
# of expected passes 16453
# of unexpected failures 483
# of expected failures 9
# of known failures 28
# of unresolved testcases 17
# of untested testcases 41
# of unsupported tests 25
</pre>
<p>
There are actually some regressions and a set of tests that fails probably due to environment differences like lack of gfortran at hand.
<p>
Full log is
<a href="http://netbsd.org/~kamil/gdb/test-gdb.base-8.0.50.20170905-git.gdb.sum">here</a>
<p>
<h1>GSoC Mentoring</h1>
<p>
The Google Summer of Code programme reached the end.
My mentees wrote successfully their final reports:
<ul>
<li><a href="//blog.netbsd.org/tnf/entry/enchancing_syzkaller_support_for_netbsd">
Enchancing Syzkaller Support for NetBSD, Part 3</a></li>
<li><a href="//blog.netbsd.org/tnf/entry/adapting_triforceafl_for_netbsd_part2">
Adapting TriforceAFL for NetBSD, Part 3</a></li>
</ul>
<p>
I'm also mentoring the AFL+KCOV work by Maciej Grochowski.
Maciej will visit EuroBSDCon-2019 and
<a href="https://2019.eurobsdcon.org/talk-speakers/#AFL">speak</a> about his work.
<p>
<h1>Add methods for setting and getting the thread name</h1>
<p>
I've reached out to the people from standards bodies and I'm working on defining the standard approach for setting and getting the thread name.
I have received a proper ID of my proposal and I'm now supposted to submit the text in either PDF or HTML format.
<p>
This change will allow to manage the thread name with an uniform interface on all comforming platforms.
<p>
<h1>Plan for the next milestone</h1>
<p>
Keep enhancing GDB support.
Keep detecting ptrace(2) bugs and addressing them.
<p>
<h2>This work was sponsored by The NetBSD Foundation.</h2>
<p>
The NetBSD Foundation is a non-profit organization and welcomes any
donations to help us continue funding projects and services to the
open-source community. Please consider visiting the following URL to
chip in what you can:
<p>
<a href="http://netbsd.org/donations/#how-to-donate">http://netbsd.org/donations/#how-to-donate</a>https://blog.netbsd.org/tnf/entry/enchancing_syzkaller_support_for_netbsd1Enchancing Syzkaller Support for NetBSD, Part 3Kamil Rytarowski2019-08-27T19:37:58+00:002019-08-27T19:37:58+00:00<p>Prepared by Siddharth Muralee(@R3x) as a part of Google Summer of Code’19</p>
<p>As a part of Google Summer of Code’19, I am working on improving the support for Syzkaller kernel fuzzer. Syzkaller is an unsupervised coverage-guided kernel fuzzer, that supports a variety of operating systems including NetBSD.</p>
<p>You can take a look through the <a href= "//blog.netbsd.org/tnf/entry/enhancing_syzkaller_support_for_netbsd" >first report</a> to see the initial changes that we made and you can look at the <a href="//blog.netbsd.org/tnf/entry/enchancing_syzkaller_support_for_netbsd" >second report</a> to read about the initial support we added for fuzzing the network stack.</p>
<p>This report details the work done during the final coding period where the target was to improve the support for fuzzing the filesystem stack.</p>
<p>Filesystem fuzzing is a relatively less explored area. Syzkaller itself only has filesystem fuzzing support for Linux.</p><p>Prepared by Siddharth Muralee(@R3x) as a part of Google Summer of Code’19</p>
<p>As a part of Google Summer of Code’19, I am working on improving the support for Syzkaller kernel fuzzer. Syzkaller is an unsupervised coverage-guided kernel fuzzer, that supports a variety of operating systems including NetBSD.</p>
<p>You can take a look through the <a href= "//blog.netbsd.org/tnf/entry/enhancing_syzkaller_support_for_netbsd" >first report</a> to see the initial changes that we made and you can look at the <a href="//blog.netbsd.org/tnf/entry/enchancing_syzkaller_support_for_netbsd" >second report</a> to read about the initial support we added for fuzzing the network stack.</p>
<p>This report details the work done during the final coding period where the target was to improve the support for fuzzing the filesystem stack.</p>
<p>Filesystem fuzzing is a relatively less explored area. Syzkaller itself only has filesystem fuzzing support for Linux.</p>
<h2>Analysis of the existing Linux setup</h2>
<p>Filesystems are more complex fuzzing target than standalone system calls. To fuzz Filesystems we do have a standard operation like mount which comes with system call vector and an additional binary image of the filesystem itself. While normal syscalls generally have a size of a few bytes, sizes of real world Filesystem images is in order of Gigabytes or larger, however for fuzzing minimal size can be used which is in order of KB-MB. Since syzkaller uses a technique called as mutational fuzzing - where it mutates random parts of the input (according to specified guidelines), having a large input size causes delay due to higher I/O time.</p>
<p>Syzkaller deals with large images by disassembling them to non-zero chunks of the filesystem image. Syzkaller extracts the non-zero chunks and their offsets and stores it as separate segments and just before execution it writes all the chunks into the corresponding offsets - generating back the new/modified image.</p>
<h2>Porting it to NetBSD</h2>
<p>As an initial step towards filesystem fuzzing we decided to port the existing Linux approach of creating random segments to NetBSD. There are a few differences between the mounting process in both the operating systems - the most significant of them being the difference in the arguments to mount(2).</p>
<p><b>Linux:</b></p>
<p>int mount(const char *source, const char *target, const char *filesystemtype, unsigned long mountflags, const void *data);</p>
<p><i>The data argument is interpreted by the different filesystems. Typically it is a string of comma-separated options understood by this filesystem.</i>
<i>mount(8) - shows possible arguments for each of the filesystem.</i></p>
<p>possible options for xfs filesystem in linux : </p>
<pre>
wsync, noalign, swalloc, nouuid, mtpt, grpid, nogrpid, bsdgroups,
sysvgroups,norecovery, inode64, inode32, ikeep, noikeep,
largeio, nolargeio, attr2, noattr2, filestreams, quota,
noquota, lazytime, nolazytime, usrquota, grpquota, prjquota,
uquota, gquota, pquota, uqnoenforce, gqnoenforce, pqnoenforce,
qnoenforce, discard, nodiscard, dax, barrier, nobarrier, logbufs,
biosize, sunit, swidth, logbsize, allocsize, logdev, rtdev
</pre>
<p><b>NetBSD:</b></p>
<p>Int mount(const char *type, const char *dir, int flags, void *data, size_t data_len);</p>
<p><i>The argument data describes the file system object to be mounted, and is data_len bytes long. data is a pointer to a structure that contains the type specific arguments to mount.</i></p>
<p>For FFS (one of the most common filesystems for NetBSD) - the arguments look like : </p>
<pre>
struct ufs_args {
char *fspec; /* block special file to mount */
};
</pre>
<p>Currently, we have a pseudo syscall <b>syz_mount_image</b> which does the job of writing the mutated chunks of the filesystem into a file based on their offsets and later configuring the loop device using vndconfig(8) and mounting the filesystem image using mount(8).</p>
<h2>Analysis of the current approach</h2>
<p>One way to create mountable filesystems is to convert an existing filesystem image into a syzkaller pseudo grammar representation and then add it to the corpus so that syzkaller uses it for mutation and we have a proper image.</p>
<p>Some of the noted issues with syzkaller approach (as noted in <a href="https://taesoo.kim/pubs/2019/xu:janus.pdf">"Fuzzing File Systems via Two-Dimensional Input Space Exploration</a>) : </p>
<li>Lack of metadata knowledge - This may lead to corruption of filesystem specific aspects such as checksums.</li>
<li>Lack of Context awareness - Syzkaller isn't aware of the status of the filesystem image after a few operations are performed on it.</li>
<h2>Steps Forward</h2>
<p>We also spent some time researching possible options to solve the existing issues and developing an approach that would give us better results.</p>
<h3>Image mutator approach</h3>
<p>One possible way forward is to actually use a seed image (a working filesystem image) and write a mutator which would be aware of all the metadata in the image. The mutator should be also be able to recreate metadata components such as the checksum so that the image is mountable.</p>
<p>An existing implementation of such a mutator is JANUS which is a filesystem mutator written for Linux with inspiration from fsck.</p>
<h3>Grammar based approach</h3>
<p>Syzkaller uses a pseudo-formal grammar for representing arguments to syscalls. This grammar can also be modified to actually be able to properly generate filesystem images. </p>
<p>Writing grammar to represent a filesystem image is quite a daunting task and we are not yet sure if it is possible but it is the approach that we have planned to take up as of now.</p>
<p>Proper documentation detailing the structure of a filesystem image is rather scarce which has led me to actually go through filesystem code to figure out the type, uses and limits of a certain filesystem image. This data then has to be converted to syzkaller representation to be used for fuzzing.</p>
<p>One advantage of writing a grammar that would be able to generate mountable images is that we would be able to get more coverage than fuzzing with a seed image, since we are also creating new images instead of just mutating the same image.</p>
<p>I am currently working on learning the internals of FFS and trying to write a grammar definition which can properly generate filesystem images.</p>
<h2> Miscellaneous Work </h2>
<p> Meanwhile, I have also been working in parallel on improving the existing state of Syzkaller.</p>
<h3> Add kernel compiled with KUBSAN for fuzzing </h3>
<p> So far we only used a kernel compiled with KCOV and KASAN for fuzzing with syzkaller. We also decided to add support for syzkaller building a kernel with KUBSAN and KCOV. This would help us have an another dimension in the fuzzing process.</p>
<p> This required some changes in the build config. We had to remove the hardcoded kernel config and add support for building a kernel with a config passed to the fuzzer. This move would also help us to easily add support for upcoming sanitizers such as KMSAN.</p>
<h3>Improve syscall descriptions</h3>
<p>Improving system call descriptions is a constant ongoing work - I recently added support for fuzzing syscalls such as mount, fork and posix_spawn.</p>
<p>We are also planning to add support for fuzzing device drivers soon.</p>
<h2>Relevant Links</h2>
<li><a href="https://syzkaller.appspot.com/netbsd"> Syzkaller Dashboard for NetBSD </a></li>
<li><a href="https://github.com/google/syzkaller">Syzkaller repository on Github</a></li>
<li><a href="https://github.com/google/syzkaller/tree/master/docs/netbsd/README.md"> NetBSD docs on setting up syzkaller</a></li>
<li><a href="https://github.com/R3x/syzkaller/tree/netbsd-gsoc-19">GSoC'19 proof of work repository</a></li>
<h2>Summary</h2>
<p>We have managed to meet most of the goals that we had planned for the GSoC project. Overall, I have had a wonderful summer with the NetBSD foundation and I look forward to working with them to complete the project.</p>
<p>Last but not least, I want to thank my mentors, @kamil and @cryo for their useful suggestions and guidance. I also thank Maciej for his insight and guidance which was very fundamental during the course of the project. I would also like to thank Dmitry Vyukov, Google for helping with any issues faced with regard to Syzkaller. Finally, thanks to Google to give me a good chance to work with NetBSD community.</p>
https://blog.netbsd.org/tnf/entry/getting_the_gnu_gdbserver_toGetting the GNU gdbserver to workKamil Rytarowski2019-08-12T05:03:50+00:002019-08-12T05:46:47+00:00A number of the remaining reported ptrace(2) bugs are GDB related.
The previous support for GDB in NetBSD was in need for refreshment,
as it had no support for gdbserver capabilities.
The GDB Server is an execution mode of the debugger, which spawns a dedicated process that interacts with
its tracee.
The process then establishes a link (socket, serial, ...) with the GDB client that is controlled by a programmer.
<p>
As NetBSD-9 has finally branched and I keep receiving requests
to finish the integration of LLVM sanitizers, I have pushed this task forward too.
I have also completed a few leftover tasks from my previous months
that still needed fixes.A number of the remaining reported ptrace(2) bugs are GDB related.
The previous support for GDB in NetBSD was in need for refreshment,
as it had no support for gdbserver capabilities.
The GDB Server is an execution mode of the debugger, which spawns a dedicated process that interacts with
its tracee.
The process then establishes a link (socket, serial, ...) with the GDB client that is controlled by a programmer.
<p>
As NetBSD-9 has finally branched and I keep receiving requests
to finish the integration of LLVM sanitizers, I have pushed this task forward too.
I have also completed a few leftover tasks from my previous months
that still needed fixes.
<p>
<h1>GNU GDB</h1>
<p>
Originally the GDB debugger was meant for local debugging.
However with the request of tracing remote setups, it has been extended
to remote debugging capabilities and this move standardized
the protocol for many debuggers.
<p>
In fact the native / local / non-remote debugging is redundant with the
remote capabilities, however this blog entry is not the right place to discuss
the technical merits in comparision between them trying to convince someone.
What matters: the developers of LLDB (LLVM debugger) already
removed their local debugging variation (for Linux) and implements
only the remote process plugin.
The NetBSD support for LLDB started with this
mode of copying the Linux approach.
Bypassing no longer exists for Linux local-debugging-only support
plugin.
<P>
Inside the GNU world, the transition from local process debugging to
remote process debugging is progressing slowly. In the current state of affairs
there is a need to support two separate (however whenever possible, with code
sharing) plugins for each OS.
Also, there is intention
from some GNU debugger developers
to completely drop the native-only process plugin to debugging code.
The state in NetBSD one month ago
handled only local-debugging
and no code for remote process debugging.
The state is the same with other BSD derived systems (FreeBSD, DragonFlyBSD...)
and this state was in general kept alive with a reduced
set of features compared to Linux.
<p>
I have decided to refresh the GDB support for NetBSD as this is still
the primary debugger on NetBSD (LLDB support is still work-in-progress)
for two primary reasons:
<ul>
<li>Slight ptrace(2) changes in the kernel code can alter support in GDB/NetBSD.
With proper NetBSD handling in GDB and ability to run its regression tests
the fallout will be minimized.</li>
<li>Certain ptrace(2) reported bugs are in fact specific GDB bugs.
In order to close them as resolved I need to revamp and validate the GDB support.</li>
</ul>
<p>
I have managed to get a basic GDB client/server session to work for tracing
a simple application.
The following features have been implemented targetting as of now NetBSD/amd64:
<ul>
<li>creating inferior (debuggee in GDB terminology)</li>
<li>attaching</li>
<li>killing</li>
<li>detaching</li>
<li>resuming (with optional: single step, signal, resuming single thread)</li>
<li>waiting</li>
<li>checking whether a thread is alive</li>
<li>fetching registers</li>
<li>storing registers</li>
<li>reading memory</li>
<li>writing memory</li>
<li>requesting interruption</li>
<li>auxv reading</li>
<li>software breakpoint support</li>
<li>hardware single step support</li>
<li>svr4 ELF loader tracing</li>
<li>retrieving exec file name from pid</li>
<li>reading thread name</li>
</ul>
What is still missing in gdbserver for NetBSD/amd64:
<ul>
<li>i386 support</li>
<li>multilib support amd64/i386</li>
<li>reading TLS BASE (x86) register (there is still missing a ptrace(2) interface)</li>
<li>EXEC events handling improvement in the generic code (that maps Linux restricted semantics only)</li>
<li>hardware watchpoint/breakpoint support</li>
<li>FPU support</li>
<li>FORK, VFORK, POSIX_SPAWN events</li>
<li>reading loadmap</li>
<li>research for other features: multiprocess, mulifs, async, btrace, tracepoints, new gdb connections..</li>
</ul>
<p>
The main difficulty was with overly elaborated version of Linux proccess tracing in GDB
that contains various workaround for various kernel bugs, handles differences between behavior
of the Linux kernel (like different set of ptrace(2) calls or.. swapped arguments) between CPUs..
Finally, I have decided to base my work on unfortunately dated (and probably broken today)
gdbserver for lynxos/i386 and then keep adding missing features that are implemented for Linux.
I have passed most of the past month on debugging spurious crashes and anomalies in my GDB remote port to NetBSD.
After much work, I have finally managed to get gdbserver to work and perform successful
operations of spawning a process, inserting a breakpoint, single-stepping, detaching
and running a process until its natural completion.
<p>
I
plan to validate the native
tracing support for NetBSD, checking whether
it needs upgrading and planning to run the
GDB regression tests as soon as possible.
<p>
<h1>LLVM changes</h1>
<p>
I have updated the list of ioctl(2) operations to the state as of 9.99.3 (+36 ioctls).
The support for NVMM ioctl(2)s has been enabled and restricted as of now to amd64 only.
<p>
I have also finally managed to resolve the crash of dynamic (as DSO library)
Address Sanitizer that blocked the integration with base in Januray 2019.
My original suspect on the root cause was finally verified to be false
(mismatch in the source code files composing .so libraries).
The real reason to ship with broken dynamic ASan was inability to reuse
the same implementation of TSD (Thread-Specific Data) in asan-dynamic.
The static (default in LLVM) version of ASan can use in early initialization
TLS (Thread-Local Storage), while TSD (Thread-Specific Data) is broken.
It happened that the state with dynamic ASan is inversed and TLS breaks and TSD works.
<p>
I have also landed build rules for LLVM sanitizers into src/, for:
<ul>
<li>asan</li>
<li>ubsan</li>
<li>tsan</li>
<li>msan</li>
<li>xray</li>
<li>safestack</li>
<li>libfuzzer</li>
</ul>
These features are targetting in all the supported variations NetBSD/amd64.
The build rules are still not hooked into the MKLLVM=yes build as I intend to
backport the needed patches to llvm-8 enhancements from llvm-HEAD and then
reimport that snapshot into the NetBSD's distribution, avoiding downstream patches.
<p>
<h1>MAXRSS changes</h1>
<p>
During the work on libfuzzer last year for GSoC-2018 one of the changes that
landed the kernel was the restoration of the MAXRSS option.
MAXRSS presents the maxiumum resident set size of the process during its lifetime.
In order to finalize this work, there was still need to enable printing it in ps(1).
I have pushed a kernel fix to update the RSS related values for sysctl(3) query of
a process statistics and with help of Krzysztof Lasocki reenabled printing MAXRSS,
IDRSS (integral unshared data), ISRSS (integral unshared stack) and IXRSS
(integral shared memory size) in ps(1).
These values were commented out in the ps(1) program since the inception of
NetBSD, the first registered commit.
<p>
<pre>
$ ps -O maxrss,idrss,isrss,ixrss|head -3
PID MAXRSS IDRSS ISRSS IXRSS TTY STAT TIME COMMAND
910 5844 20 12 216 pts/0 Is 0:00.01 -ksh
1141 5844 20 12 152 pts/0 I+ 0:00.01 mg event.h
</pre>
<p>
These change have been backported to the NetBSD-9 branch and will land 9.0.
Welcome back!
<p>
<h1>Plan for the next milestone</h1>
<p>
Start executing GDB regression tests.
Keep enhancing GDB support.
Keep detecting ptrace(2) bugs and addressing them.
<p>
<h2>This work was sponsored by The NetBSD Foundation.</h2>
<p>
The NetBSD Foundation is a non-profit organization and welcomes any
donations to help us continue funding projects and services to the
open-source community. Please consider visiting the following URL to
chip in what you can:
<p>
<a href="http://netbsd.org/donations/#how-to-donate">http://netbsd.org/donations/#how-to-donate</a>