GSoC 2011 roundup: Add kqueue support to GIO

As the Google Summer of Code 2011 (GSoC 2011) program concludes, we will be running a series of articles detailing the results of the projects mentored by The NetBSD Foundation.

Today's turn is the summary of Dmitry Matveev's project, "Add kqueue support to GIO", for which I was the mentor.

Dmitry reports himself:

The goals of my GSoC 2011 project were:

1. To integrate kqueue into GIO and to provide fast file monitoring for Glib/GTK+-powered applications (Nautilus and etc).
2. To implement an inotify emulation library on the top of kqueue and to make porting inotify applications on BSDs easier.

During the first stage of the project in May, I learned the Glib internals to understand how to integrate a kqueue-based monitoring into the existing Glib event loop (GMainLoop). I wrote a detailed post about this problem. In the end, I decided to use the Mac OS X approach of having a separate thread for kqueue, and convert and dispatch notifications from this thread into Glib's event loop. The remaining part was almost trivial and the first ever kqueue-powered GFileMonitor began working in the beginning of June.

On June 20th, I posted the GIO kqueue backend sources to the Glib developers for a review, and got some valuable comments from them.

The second part of the project was even more breathtaking. I had to implement inotify for the BSDs using kqueue. The main problem I faced is that inotify provides a lot of convenient events, mostly about directory content changes, because inotify was originally designed explicitly for file system monitoring. Kqueue is a much more generic interface and does not provide some events that inotify does. Juggling with inodes to calculate the directory diff and to determine if any files were overwritten in some sophicated scenarios is an unforgettable experience! The good thing is that I got a good chance to sharpen my skills in concurrent programming.

At the beginning of September, the library was 95% complete and was covered by more than 60 tests. Also, there was a pleasant surprise when I received some patches for FreeBSD support just a few days after the program's end. There was also a first success story — I was told that the incrond application was linked with the library and ran well on FreeBSD! Libinotify is already available in NetBSD and FreeBSD ports collections.

Today I am continuing fixing sad bugs (the saddest one has been found in the GIO backend just on this week) and planning further improvements to both the backend and the library. Algorithms used in libinotify are not very optimal, so I think I can make them work faster. After some testing in pkgsrc I will try to push my Glib patches to upstream.

I would like to thank Dan Winship and Colomban Wendling for the code reviews, Stanislav Sedov for the FreeBSD port, Baptiste Daroussin for problem reports, Thomas Klausner (wiz@) for the libinotify packaging, Petra Zeidler (spz@) and, especially, my mentor Julio Merino (jmmv@) for keeping me organized on work during the program and helping me with difficult troubles. I could not have completed the project without the help from all of you.

It fills me with joy to see that the project reached the desired goals in the expected timeframe. With these changes in place, many desktop applications and some server applications will offer a nicer user experience without requiring complex application-specific patches.

Congratulations Dmitry!