GSoC Reports: Benchmarking NetBSD, third evaluation report

This report was written by Apurva Nandan as part of Google Summer of Code 2020.

Introduction

This blog post is in continuation of GSoC Reports: Benchmarking NetBSD, first evaluation report and GSoC Reports: Benchmarking NetBSD, second evaluation report blogs, and describes my progress in the final phase of GSoC 2020 under The NetBSD Foundation.

In the third phase, I upgraded to the latest stable version Phoronix Test Suite (PTS) 9.8.0 in pkgsrc-wip, resolved the TODOs and created patches for more test-profiles to fix their installation and runtime errors on NetBSD-current.

Progress in the third phase of GSoC

wip/phoronix-test-suite TODO and update

As a newer stable version of the Phoronix Test Suite was available in upstream, I upgraded the Phoronix Test Suite from version 9.6.1 to 9.8.0 in pkgsrc-wip and is available as wip/phoronix-test-suite. You can have a look at the PTS Changelog to know about the improvements between these two versions.

To get the package ready for merge in pkgsrc upstream, I also resolved the pkgsrc-wip TODOs.

pkgsrc-wip commits:

• wip/phoronix-test-suite: update phoronix-test-suite to 9.8.0
• Resolved the pending TODOs

If any new problems are encountered, please document them in wip/phoronix-test-suite/TODO file and/or contact me.

Testing of automated benchmarking framework

I had been assigned a remote testing machine having Intel 6138 dual processor, 40 cores, 80 threads, 192GB of RAM. I spent time reproducing my automated framework i.e., Phoromatic-Anita Integration on the machine. I was able to reproduce the integration framework working without networking configuration, but the network bridge needs to be setup on the remote machine and the integration script to be tested with it. I shall continue this task in the post-GSoC period.

Benchmarking Results

I also performed benchmarking of NetBSD-9 amd64 native installation by running 50 test-profiles on a remote machine assigned to me by mentors and uploaded the benchmark results to OpenBenchmarking.org at:

• https://openbenchmarking.org/result/2008287-NE-3966268951

Test-profile debugging

I then continued the task of maintaining/porting test-profiles and fixed the following test-profiles:

Timed FFmpeg Compilation

This test times how long it takes to build FFmpeg. This test is part of Processor Test category.

Original Test-profile:

https://openbenchmarking.org/test/pts/build-ffmpeg

Patched Test-profile:

https://github.com/apurvanandan1997/pts-test-profiles-dev/tree/master/build-ffmpeg-1.0.1

Commit:

• Replace the make -> gmake for compatibility with NetBSD

Compile Bench

Compilebench tries to age a filesystem by simulating some of the disk IO common in creating, compiling, patching, stating and reading kernel trees. It indirectly measures how well filesystems can maintain directory locality as the disk fills up and directories age. This test is part of Disk Test category.

Original Test-profile:

https://openbenchmarking.org/test/pts/compilebench

Patched Test-profile:

https://github.com/apurvanandan1997/pts-test-profiles-dev/tree/master/compilebench-1.0.2

Commit:

• Replaced python2 with NetBSD style naming python2.7

Timed MAFFT Alignment

This test performs an alignment of 100 pyruvate decarboxylase sequences. This test is part of Processor Test category.

Original Test-profile:

https://openbenchmarking.org/test/pts/mafft

Patched Test-profile:

https://github.com/apurvanandan1997/pts-test-profiles-dev/tree/master/mafft-1.5.0

Commits:

• Replaced the make -> gmake for compatibility with NetBSD
• Patch for replacing /bin/bash interpreter with /usr/pkg/bin/bash

Future Plans

This officially summarizes my GSoC project: Benchmark NetBSD, and my end goal of the project that is to integrate Phoronix Test Suite with NetBSD and Anita for automated benchmarking is complete and its deployment on benchmark.NetBSD.org will be continued to be worked on with the coordination of moderators and merging the wip of Phoronix Test Suite 9.8.0 will be done by the pkgsrc maintainers in next days.

I want to thank my mentors and the NetBSD community without whose constant support I wouldn't have achieved the goals.