GSoC 2018 Reports: Configuration files versioning in pkgsrc, part 3: remote repositories (SVN and Mercurial)


August 26, 2018 posted by Thomas Klausner

In GSoC 2018 Reports: Configuration files versioning in pkgsrc, part 2: remote repositories (git and CVS) structural changes to the configuration tracking feature of pkgsrc got introduced, along with a short walkthrough in using remote Git and CVS repositories.

This third post will introduce SVN and Mercurial support, leaving configuration deployment for the final entry. Stay tuned!

SVN and remotes

Subversion, when used locally, needs to be set this way, as you can expect:

pkghost# cat /usr/pkg/etc/pkg_install.conf                                                                    
VCSTRACK_CONF=yes                                                                                            
VCS=svn                                                                                                       
VCSDIR=/var/svnconfdir                                                                                        
VCSAUTOMERGE=yes 

A local repository will be created at VCSDIR/localsvn, files will get extracted and checkedin at VCSDIR/defaults, VCSDIR/automerged, VCSDIR/user

pkghost# make   

[...]
WARNING: To fix run: `/usr/pkg/sbin/pkg_admin -K /var/db/pkg fetch-pkg-vulnerabilities'.                      
===> Installing for spamd-20060330nb5                                                                         
=> Generating pre-install file lists                                                                          
=> Creating installation directories                                                                          
/usr/bin/install -c -o root -g wheel -m 644 /root/pkgsrc/mail/spamd/work/spamd-20060330/man/spamd.conf.5  /ro5
/usr/bin/install -c -o root -g wheel -m 644 /root/pkgsrc/mail/spamd/work/spamd-20060330/spamd/spamd.8  /root/8
/usr/bin/install -c -o root -g wheel -m 644 /root/pkgsrc/mail/spamd/work/spamd-20060330/spamd-setup/spamd-set8
/usr/bin/install -c -o root -g wheel -m 644 /root/pkgsrc/mail/spamd/work/spamd-20060330/spamdb/spamdb.8  /roo8
/usr/bin/install -c -o root -g wheel -m 644 /root/pkgsrc/mail/spamd/work/spamd-20060330/spamlogd/spamlogd.8  8
/usr/bin/install -c -s -o root -g wheel -m 755 /root/pkgsrc/mail/spamd/work/spamd-20060330/spamd-setup/spamd-c
/usr/bin/install -c -s -o root -g wheel -m 755 /root/pkgsrc/mail/spamd/work/spamd-20060330/spamd/spamd  /rootc
/usr/bin/install -c -s -o root -g wheel -m 755 /root/pkgsrc/mail/spamd/work/spamd-20060330/spamdb/spamdb  /ron
/usr/bin/install -c -s -o root -g wheel -m 755 /root/pkgsrc/mail/spamd/work/spamd-20060330/spamlogd/spamlogd c
/usr/bin/install -c -o root -g wheel -m 644 /root/pkgsrc/mail/spamd/work/spamd-20060330/etc/spamd.conf /root/d
=> Automatic manual page handling                                                                             
=> Generating post-install file lists                                                                         
=> Checking file-check results for spamd-20060330nb5                                                          
=> Creating binary package /root/pkgsrc/mail/spamd/work/.packages/spamd-20060330nb5.tgz                       
===> Building binary package for spamd-20060330nb5                                                            
=> Creating binary package /root/pkgsrc/packages/All/spamd-20060330nb5.tgz                                    
===> Installing binary package of spamd-20060330nb5                                                           
REGISTER /var/svnconfdir/defaults//usr/pkg/etc/spamd.conf                                                     
spamd-20060330nb5: copying /usr/pkg/share/examples/spamd/spamd.conf to /usr/pkg/etc/spamd.conf                
Conf commit: pkgsrc: add spamd-20060330nb5                                                                    
===========================================================================                                   
The following files should be created for spamd-20060330nb5:                                                  
                                                                                                              
        /etc/rc.d/pfspamd (m=0755)                                                                            
            [/usr/pkg/share/examples/rc.d/pfspamd]                                                            
                                                                                                              
===========================================================================                                   
===========================================================================                                   
$NetBSD: MESSAGE,v 1.1.1.1 2005/06/28 12:43:57 peter Exp $                                                    
                                                                                                              
Don't forget to add the spamd ports to /etc/services:                                                         
                                                                                                              
spamd           8025/tcp                # spamd(8)                                                            
spamd-cfg       8026/tcp                # spamd(8) configuration                                              
                                                                                                              
===========================================================================                                   

pkghost# ls -lah /var/svnconfdir/localsvn/                                                                    
total 3.2K                                                                                                    
drwxr-xr-x  6 pkgvcsconf  pkgvcsconf  512B Aug  4 07:22 .                                                     
drwx------  6 pkgvcsconf  pkgvcsconf  512B Aug  4 07:22 ..                                                    
-rw-r--r--  1 pkgvcsconf  pkgvcsconf  246B Aug  4 07:22 README.txt                                            
drwxr-xr-x  2 pkgvcsconf  pkgvcsconf  512B Aug  4 07:22 conf                                                  
drwxr-sr-x  6 pkgvcsconf  pkgvcsconf  512B Aug  4 07:22 db                                                    
-r--r--r--  1 pkgvcsconf  pkgvcsconf    2B Aug  4 07:22 format                                                
drwxr-xr-x  2 pkgvcsconf  pkgvcsconf  512B Aug  4 07:22 hooks                                                 
drwxr-xr-x  2 pkgvcsconf  pkgvcsconf  512B Aug  4 07:22 locks

pkghost# vi /usr/local/etc/spamd.conf
whitelist:\ 
        :white:\
        :method=file:\
        :file=/var/mail/whitelist.txt:                                                                        
/usr/pkg/etc/spamd.conf: 86 lines, 2767 characters                                                            
                                                  .                                                           
pkghost# make replace
[...]

and so on, you can come up with what follows. Let's test a remote repository, starting with preparations on the server:

$ hostname                                                                                                    
vers                                                                                                          
$ cd /usr/home/vcs   
$ svnadmin create svnremote

That's it, no branches or releases are used, so you don't need to create any further structure on the repository, the +VERSIONING script will take care of the remaining bits (you should migrate data from a local repository to the remote one, if you desire so, before running package installations).

Use a similarly structured REMOTEVCS URI: svn+ssh://vcs@192.168.100.112/usr/home/vcs/svnremote

(as with CVS, you can also run it as a standalone network server without ssh transport and authentication)

pkghost# cat /usr/pkg/etc/pkg_install.conf                                                                    
VCSTRACK_CONF=yes                                                                                            
VCS=svn                                                                                                       
VCSDIR=/var/svnwrkremote                                                                                      
VCSAUTOMERGE=yes                                                                                              
REMOTEVCS=svn+ssh://vcs@192.168.100.112/usr/home/vcs/svnremote   
pkghost# make                                                                                                 

[...]

pkghost# make install                                                                                         
=> Bootstrap dependency digest>=20010302: found digest-20160304                                               
===> Skipping vulnerability checks.                                                                           
WARNING: No /var/db/pkg/pkg-vulnerabilities file found.                                                       
WARNING: To fix run: `/usr/pkg/sbin/pkg_admin -K /var/db/pkg fetch-pkg-vulnerabilities'.                      
===> Installing for spamd-20060330nb6                                                                         
=> Generating pre-install file lists                                                                          
=> Creating installation directories                                                                          
/usr/bin/install -c -o root -g wheel -m 644 /root/pkgsrc/mail/spamd/work/spamd-20060330/man/spamd.conf.5  /ro5
/usr/bin/install -c -o root -g wheel -m 644 /root/pkgsrc/mail/spamd/work/spamd-20060330/spamd/spamd.8  /root/8
/usr/bin/install -c -o root -g wheel -m 644 /root/pkgsrc/mail/spamd/work/spamd-20060330/spamd-setup/spamd-set8
/usr/bin/install -c -o root -g wheel -m 644 /root/pkgsrc/mail/spamd/work/spamd-20060330/spamdb/spamdb.8  /roo8
/usr/bin/install -c -o root -g wheel -m 644 /root/pkgsrc/mail/spamd/work/spamd-20060330/spamlogd/spamlogd.8  8
/usr/bin/install -c -s -o root -g wheel -m 755 /root/pkgsrc/mail/spamd/work/spamd-20060330/spamd-setup/spamd-c
/usr/bin/install -c -s -o root -g wheel -m 755 /root/pkgsrc/mail/spamd/work/spamd-20060330/spamd/spamd  /rootc
/usr/bin/install -c -s -o root -g wheel -m 755 /root/pkgsrc/mail/spamd/work/spamd-20060330/spamdb/spamdb  /ron
/usr/bin/install -c -s -o root -g wheel -m 755 /root/pkgsrc/mail/spamd/work/spamd-20060330/spamlogd/spamlogd c
/usr/bin/install -c -o root -g wheel -m 644 /root/pkgsrc/mail/spamd/work/spamd-20060330/etc/spamd.conf /root/d
=> Automatic manual page handling                                                                             
=> Generating post-install file lists                                                                         
=> Checking file-check results for spamd-20060330nb6                                                          
=> Creating binary package /root/pkgsrc/mail/spamd/work/.packages/spamd-20060330nb6.tgz                       
===> Building binary package for spamd-20060330nb6                                                            
=> Creating binary package /root/pkgsrc/packages/All/spamd-20060330nb6.tgz                                    
===> Installing binary package of spamd-20060330nb6                                                           
svn: E155007: '/var/svnwrkremote/defaults' is not a working copy                                              
Committing transaction...                                                                                     
Committed revision 1.                                                                                         
Checked out revision 1.                                                                                       
Committing transaction...                                                                                     
Committed revision 2.                                                                                         
Checked out revision 2.                                                                                       
Committing transaction...                                                                                     
Committed revision 3.                                                                                         
Checked out revision 3.                                                                                       
A         usr                                                                                                 
A         pkg                                                                                                 
A         etc                                                                                                 
A         spamd.conf                                                                                          
REGISTER /var/svnwrkremote/defaults//usr/pkg/etc/spamd.conf                                                   
spamd-20060330nb6: copying /usr/pkg/share/examples/spamd/spamd.conf to /usr/pkg/etc/spamd.conf                
Adding         usr                                                                                            
Adding         usr/pkg                                                                                        
Adding         usr/pkg/etc                                                                                    
Adding         usr/pkg/etc/spamd.conf                                                                         
Transmitting file data .done                                                                                  
Committing transaction...                                                                                     
Committed revision 4.                                                                                         
Conf commit: pkgsrc: add spamd-20060330nb6                                                                    
===========================================================================                                   
The following files should be created for spamd-20060330nb6:                                                  
                                                                                                              
        /etc/rc.d/pfspamd (m=0755)                                                                            
            [/usr/pkg/share/examples/rc.d/pfspamd]                                                            
                                                                                                              
===========================================================================                                   
===========================================================================                                   
$NetBSD: MESSAGE,v 1.1.1.1 2005/06/28 12:43:57 peter Exp $                                                    
                                                                                                              
Don't forget to add the spamd ports to /etc/services:                                                         
                                                                                                              
spamd           8025/tcp                # spamd(8)                                                            
spamd-cfg       8026/tcp                # spamd(8) configuration                                              
                                                                                                              
===========================================================================                                   
pkghost# 

since I didn't show it with CVS, let's try again some simulated file edits:

pkghost# tail -n 5 /usr/pkg/etc/spamd.conf                                                                    
#                                                                                                             
#whitelist:\                                                                                                  
#       :white:\                                                                                              
#       :method=file:\                                                                                        
#       :file=/var/mail/whitelist.txt:                                                                        
pkghost# vi /usr/pkg/etc/spamd.conf; tail -n 5 /usr/pkg/etc/spamd.conf 
/usr/pkg/etc/spamd.conf: 86 lines, 2767 characters                                                            
                                                  .                                                           
#                                                                                                             
whitelist:\                                                                                                   
        :white:\                                                                                              
        :method=file:\                                                                                        
        :file=/var/mail/whitelist.txt:  
=> Bootstrap dependency digest>=20010302: found digest-20160304                                               
===> Skipping vulnerability checks.                                                                           
WARNING: No /var/db/pkg/pkg-vulnerabilities file found.                                                       
WARNING: To fix run: `/usr/pkg/sbin/pkg_admin -K /var/db/pkg fetch-pkg-vulnerabilities'.                      
===> Replacing for spamd-20060330nb6                                                                          
===> Updating using binary package of spamd-20060330nb6                                                       
/usr/bin/env  /usr/pkg/sbin/pkg_add -K /var/db/pkg -U -D /root/pkgsrc/mail/spamd/work/.packages/spamd-2006033z
===========================================================================                                   
The following users are no longer being used by spamd-20060330nb6,                                            
and they can be removed if no other software is using them:                                                   
                                                                                                              
        _spamd                                                                                                
                                                                                                              
===========================================================================                                   
===========================================================================                                   
The following groups are no longer being used by spamd-20060330nb6,                                           
and they can be removed if no other software is using them:                                                   
                                                                                                              
        _spamd                                                                                                
                                                                                                              
===========================================================================                                   
===========================================================================                                   
The following files are no longer being used by spamd-20060330nb6,                                            
and they can be removed if no other packages are using them:                                                  
                                                                                                              
        /usr/pkg/etc/spamd.conf                                                                               
                                                                                                              
===========================================================================                                   
REGISTER /var/svcwrkremote/defaults//usr/pkg/etc/spamd.conf                                                   
spamd-20060330nb6: /usr/pkg/etc/spamd.conf already exists                                                     
spamd-20060330nb6: attempting to merge /usr/pkg/etc/spamd.conf with new defaults!                             
Saving the currently user-installed revision to /var/svnwrkremote/user//usr/pkg/etc/spamd.conf                
A         usr                                                                                                 
A         pkg                                                                                                 
A         etc                                                                                                 
A         spamd.conf                                                                                          
Adding         usr                                                                                            
Adding         usr/pkg                                                                                        
Adding         usr/pkg/etc                                                                                    
Adding         usr/pkg/etc/spamd.conf                                                                         
Transmitting file data .done                                                                                  
Committing transaction...                                                                                     
Committed revision 5.                                                                                         
Conf commit: pkgsrc: backup user conf before attempting merge for spamd-20060330nb6                           
A    defaults/usr/pkg/etc/spamd.conf                                                                          
Export complete.                                                                                              
Merged with no conflicts. installing it to /usr/pkg/etc/spamd.conf!                                           
A         usr                                                                                                 
A         pkg                                                                                                 
A         etc                                                                                                 
A         spamd.conf                                                                                          
A    defaults/usr/pkg/etc/spamd.conf                                                                          
Export complete.                                                                                              
Revert from the last revision of /var/svcwrkremote/user//usr/pkg/etc/spamd.conf if needed                     
Adding         usr                                                                                            
Adding         usr/pkg                                                                                        
Adding         usr/pkg/etc                                                                                    
Adding         usr/pkg/etc/spamd.conf                                                                         
Transmitting file data .done                                                                                  
Committing transaction...                                                                                     
Committed revision 6.                                                                                         
Conf commit: pkgsrc: add spamd-20060330nb6                                                                    
===========================================================================                                   
The following files should be created for spamd-20060330nb6:                                                  
                                                                                                              
        /etc/rc.d/pfspamd (m=0755)                                                                            
            [/usr/pkg/share/examples/rc.d/pfspamd]                                                            
                                                                                                              
===========================================================================                                   
===========================================================================                                   
$NetBSD: MESSAGE,v 1.1.1.1 2005/06/28 12:43:57 peter Exp $                                                    
                                                                                                              
Don't forget to add the spamd ports to /etc/services:                                                         
                                                                                                              
spamd           8025/tcp                # spamd(8)                                                            
spamd-cfg       8026/tcp                # spamd(8) configuration                                              
                                                                                                              
===========================================================================  
pkghost# cd /var/svnwrkremote/                                                                                
pkghost# ls                                                                                                   
automerged      automergedfiles defaults        user    
pkghost# cd automerged  
pkghost# svn update                                                                                           
Updating '.':                                                                                                 
At revision 6.                                                                                                
pkghost# svn log                                                                                              
------------------------------------------------------------------------                                      
r6 | vcs | 2018-08-04 13:22:29 +0000 (Sat, 04 Aug 2018) | 1 line                                              
                                                                                                              
pkgsrc: add spamd-20060330nb6                                                                                 
------------------------------------------------------------------------                                      
r1 | vcs | 2018-08-04 13:19:29 +0000 (Sat, 04 Aug 2018) | 1 line                                              
                                                                                                              
initial import                                                                                                
------------------------------------------------------------------------ 

pkghost# cd $HOME/pkgsrc/mail/spamd                                                                           
pkghost# grep REVISION Makefile                                                                               
PKGREVISION=            6                                                                                     
pkghost# sed -i "s/PKGREVISION=.*6/PKGREVISION=7/g" Makefile                                                  
pkghost# grep REVISION Makefile                                                                               
PKGREVISION=7   
pkghost# make clean; make extract; sed -i "s/http/https/g" work/spamd-20060330/etc/spamd.conf
pkghost# make update
===> Skipping vulnerability checks.
WARNING: No /var/db/pkg/pkg-vulnerabilities file found.
WARNING: To fix run: `/usr/pkg/sbin/pkg_admin -K /var/db/pkg fetch-pkg-vulnerabilities'.
===> Deinstalling for spamd-20060330nb7
Running /usr/pkg/sbin/pkg_delete -K /var/db/pkg -r spamd-20060330nb6
===========================================================================

[...]

===> Installing for spamd-20060330nb7                                                                         
=> Generating pre-install file lists                                                                          
=> Creating installation directories                                                                          
/usr/bin/install -c -o root -g wheel -m 644 /root/pkgsrc/mail/spamd/work/spamd-20060330/man/spamd.conf.5  /ro5
/usr/bin/install -c -o root -g wheel -m 644 /root/pkgsrc/mail/spamd/work/spamd-20060330/spamd/spamd.8  /root/8
/usr/bin/install -c -o root -g wheel -m 644 /root/pkgsrc/mail/spamd/work/spamd-20060330/spamd-setup/spamd-set8
/usr/bin/install -c -o root -g wheel -m 644 /root/pkgsrc/mail/spamd/work/spamd-20060330/spamdb/spamdb.8  /roo8
/usr/bin/install -c -o root -g wheel -m 644 /root/pkgsrc/mail/spamd/work/spamd-20060330/spamlogd/spamlogd.8  8
/usr/bin/install -c -s -o root -g wheel -m 755 /root/pkgsrc/mail/spamd/work/spamd-20060330/spamd-setup/spamd-c
/usr/bin/install -c -s -o root -g wheel -m 755 /root/pkgsrc/mail/spamd/work/spamd-20060330/spamd/spamd  /rootc
/usr/bin/install -c -s -o root -g wheel -m 755 /root/pkgsrc/mail/spamd/work/spamd-20060330/spamdb/spamdb  /ron
/usr/bin/install -c -s -o root -g wheel -m 755 /root/pkgsrc/mail/spamd/work/spamd-20060330/spamlogd/spamlogd c
/usr/bin/install -c -o root -g wheel -m 644 /root/pkgsrc/mail/spamd/work/spamd-20060330/etc/spamd.conf /root/d
=> Automatic manual page handling                                                                             
=> Generating post-install file lists                                                                         
=> Checking file-check results for spamd-20060330nb7                                                          
=> Creating binary package /root/pkgsrc/mail/spamd/work/.packages/spamd-20060330nb7.tgz                       
===> Building binary package for spamd-20060330nb7                                                            
=> Creating binary package /root/pkgsrc/packages/All/spamd-20060330nb7.tgz                                    
===> Installing binary package of spamd-20060330nb7                                                           
REGISTER /var/svcwrkremote/defaults//usr/pkg/etc/spamd.conf                                                   
spamd-20060330nb7: /usr/pkg/etc/spamd.conf already exists                                                     
spamd-20060330nb7: attempting to merge /usr/pkg/etc/spamd.conf with new defaults!                             
A    automerged/usr/pkg/etc/spamd.conf                                                                        
Export complete.                                                                                              
Saving the currently installed revision to /var/svcwrkremote/automerged//usr/pkg/etc/spamd.conf               
Sending        usr/pkg/etc/spamd.conf                                                                         
Transmitting file data .done                                                                                  
Committing transaction...                                                                                     
Committed revision 7.                                                                                         
Conf commit: pkgsrc: backup preexisting conf before attempting merge for spamd-20060330nb7                    
A    defaults/usr/pkg/etc/spamd.conf                                                                          
Export complete.                                                                                              
Merged with no conflict. installing it to /usr/pkg/etc/spamd.conf!                                            
--- /usr/pkg/etc/spamd.conf     2018-08-04 09:01:09.536149671 +0000                                           
+++ /var/svcwrkremote/defaults//usr/pkg/etc/spamd.conf.automerge        2018-08-04 09:10:52.131955252 +0000   
@@ -15,7 +15,7 @@                                                                                             
 # may be applied to each blacklist.                                                                          
 #                                                                                                            
 # As of November 2004, a place to search for black lists is                                                  
-#     http://spamlinks.net/filter-bl.htm                                                                     
+#     https://spamlinks.net/filter-bl.htm                                                                    
 #                                                                                                            
 # Some of the URLs below point to www.openbsd.org locations.  Those                                          
 # files are likely to be mirrored to other OpenBSD www mirrors located                                       
@@ -25,45 +25,45 @@                                                                                           
 all:\                                                                                                        
        :spews1:china:korea:                                                                                  
                                                                                                              
-# Mirrored from http://spfilter.openrbl.org/data/sbl/SBL.cidr.bz2                                            
+# Mirrored from https://spfilter.openrbl.org/data/sbl/SBL.cidr.bz2                                           
 spamhaus:\                                                                                                   
        :black:\                                                                                              
        :msg="SPAM. Your address %A is in the Spamhaus Block List\n\                                          
-       See http://www.spamhaus.org/sbl and\                                                                  
-       http://www.abuse.net/sbl.phtml?IP=%A for more details":\                                              
-       :method=http:\                                                                                        
+       See https://www.spamhaus.org/sbl and\                                                                 
+       https://www.abuse.net/sbl.phtml?IP=%A for more details":\                                             
+       :method=https:\                                                                                       
        :file=www.openbsd.org/spamd/SBL.cidr.gz:                                                              
                                                                                                              
-# Mirrored from http://www.spews.org/spews_list_level1.txt                                                   
+# Mirrored from https://www.spews.org/spews_list_level1.txt                                                  
 spews1:\                                                                                                     
        :black:\                                                                                              
        :msg="SPAM. Your address %A is in the spews level 1 database\n\                                       
-       See http://www.spews.org/ask.cgi?x=%A for more details":\                                             
-       :method=http:\                                                                                        
+       See https://www.spews.org/ask.cgi?x=%A for more details":\                                            
+       :method=https:\                                                                                       
        :file=www.openbsd.org/spamd/spews_list_level1.txt.gz:                                                 
                                                                                                              
-# Mirrored from http://www.spews.org/spews_list_level2.txt                                                   
+# Mirrored from https://www.spews.org/spews_list_level2.txt                                                  
 spews2:\                                                                                                     
        :black:\                                                                                              
        :msg="SPAM. Your address %A is in the spews level 2 database\n\                                       
-       See http://www.spews.org/ask.cgi?x=%A for more details":\                                             
-       :method=http:\                                                                                        
+       See https://www.spews.org/ask.cgi?x=%A for more details":\                                            
+       :method=https:\                                                                                       
        :file=www.openbsd.org/spamd/spews_list_level2.txt.gz:                                                 
                                                                                                              
-# Mirrored from http://www.okean.com/chinacidr.txt                                                           
+# Mirrored from https://www.okean.com/chinacidr.txt                                                          
 china:\                                                                                                      
        :black:\                                                                                              
        :msg="SPAM. Your address %A appears to be from China\n\                                               
-       See http://www.okean.com/asianspamblocks.html for more details":\                                     
-       :method=http:\                                                                                        
+       See https://www.okean.com/asianspamblocks.html for more details":\                                    
+       :method=https:\                                                                                       
        :file=www.openbsd.org/spamd/chinacidr.txt.gz:                                                         
                                                                                                              
-# Mirrored from http://www.okean.com/koreacidr.txt                                                           
+# Mirrored from https://www.okean.com/koreacidr.txt                                                          
 korea:\                                                                                                      
        :black:\                                                                                              
        :msg="SPAM. Your address %A appears to be from Korea\n\                                               
-       See http://www.okean.com/asianspamblocks.html for more details":\                                     
-       :method=http:\                                                                                        
+       See https://www.okean.com/asianspamblocks.html for more details":\                                    
+       :method=https:\                                                                                       
        :file=www.openbsd.org/spamd/koreacidr.txt.gz:                                                         
                                                                                                              
 #relaydb-black:\                                                                                             
Revert from the penultimate revision of /var/svcwrkremote/automerged//usr/pkg/etc/spamd.conf if needed        
Sending        usr/pkg/etc/spamd.conf                                                                         
Transmitting file data .done                                                                                  
Committing transaction...                                                                                     
Committed revision 8.                                                                                         
Sending        usr/pkg/etc/spamd.conf                                                                         
Transmitting file data .done                                                                                  
Committing transaction...                                                                                     
Committed revision 9.                                                                                         
Conf commit: pkgsrc: add spamd-20060330nb7                                                                    
===========================================================================                                   
The following files should be created for spamd-20060330nb7:                                                  
                                                                                                              
        /etc/rc.d/pfspamd (m=0755)                                                                            
            [/usr/pkg/share/examples/rc.d/pfspamd]                                                            
                                                                                                              
===========================================================================                                   
[...]
===> Cleaning for spamd-20060330nb7                                                                           

HG and remotes

Yes, mercurial. Mercurial, like git, uses a local directory (namely .hg) located under the working directory VCSDIR, it is also used to store configuration and URIs to remote repositories. to initialize and test it (as said, no repository and vcs migrations are supported by pkgsrc itself, you should take care of migrations yourself if you want to), just set pkg_install.conf to use a local mercurial repo and install a package:

pkghost# rm /usr/pkg/etc/spamd.conf                                                                           
pkghost# /usr/pkg/sbin/pkg_delete spamd
pkghost# vi /usr/pkg/etc/pkg_install.conf; cat /usr/pkg/etc/pkg_install.conf
/usr/pkg/etc/pkg_install.conf: 4 lines, 66 characters                                                         
                                                     .                                                        
VCSTRACK_CONF=yes                                                                                            
VCS=hg                                                                                                        
VCSDIR=/var/hglocaldir                                                                                        
VCSAUTOMERGE=yes                                                                                              
pkghost# make                                                                                                 
=> Bootstrap dependency digest>=20010302: found digest-20160304                                               
===> Skipping vulnerability checks.                                                                           
WARNING: No /var/db/pkg/pkg-vulnerabilities file found.                                                       
WARNING: To fix run: `/usr/pkg/sbin/pkg_admin -K /var/db/pkg fetch-pkg-vulnerabilities'.                      
=> Checksum SHA1 OK for spamd-20060330.tar.gz

[...]

To save some space and time, let's just say that it works as expected:

[...]
Conf commit: pkgsrc: backup preexisting conf before attempting merge for spamd-20060330nb10                   
Merged with no conflict. installing it to /usr/pkg/etc/spamd.conf!                                            
--- /usr/pkg/etc/spamd.conf     2018-08-04 09:50:00.738099463 +0000                                           
+++ /var/hglocaldir/defaults//usr/pkg/etc/spamd.conf.automerge  2018-08-04 09:52:18.467604445 +0000           
@@ -15,7 +15,7 @@                                                                                             
 # may be applied to each blacklist.                                                                          
 #                                                                                                            
 # As of November 2004, a place to search for black lists is                                                  
-#     http://spamlinks.net/filter-bl.htm                                                                     
+#     https://spamlinks.net/filter-bl.htm                                                                    
 # 
 [...]
 

One nice thing about mercurial is the simplicity enabling one to clone a local repository to a remote server. The script, when using mercurial, tries exacly that, this should succeed if the remote repository is empty.

 				if ${TEST} "$_REMOTE" != "no" -a "$_REMOTE" != "NO"; then
					execute "hg clone . \"$_REMOTE\""
					execute "hg --repository \"$_VCSDIR\" push \"$_REMOTE\""
					execute "hg --repository \"$_VCSDIR\" pull \"$_REMOTE\""
                fi

let's init an empty repository on the server and let pkgsrc clone the existing files over there!

$ cd                                                                                                          
$ hostname                                                                                                    
vers                                                                                                          
$ pwd                                                                                                         
/home/vcs                                                                                                     
$ hg init pkgconftest                                                                                         
$ ls pkgconftest/                                                                                             
$ ls pkgconftest/.hg/                                                                                         
00changelog.i   requires        store                                                                         
$ 

URIs for REMOTEVCS take the following format, should you choose to use ssh instead of hg server, http or other access methods that you'll find documented on official Mercurial resources, as with svn, git and cvs:

ssh://vcs@192.168.100.112/usr/home/vcs/pkgconftest

pkghost# vi /usr/pkg/etc/pkg_install.conf; cat /usr/pkg/etc/pkg_install.conf

/usr/pkg/etc/pkg_install.conf: 5 lines, 127 characters                                                        
                                                      .                                                       
VCSTRACK_CONF=yes                                                                                            
VCS=hg                                                                                                        
VCSDIR=/var/hglocaldir                                                                                        
VCSAUTOMERGE=yes                                                                                              
REMOTEVCS=ssh://vcs@192.168.100.112/usr/home/vcs/pkgconftest                                                  

pkghost# make replace  

[...]

===> Updating using binary package of spamd-20060330nb10                                                      
/usr/bin/env  /usr/pkg/sbin/pkg_add -K /var/db/pkg -U -D /root/pkgsrc/mail/spamd/work/.packages/spamd-2006033z
===========================================================================                                   
The following users are no longer being used by spamd-20060330nb10,                                           
and they can be removed if no other software is using them:                                                   
                                                                                                              
        _spamd                                                                                                
                                                                                                              
===========================================================================                                   
===========================================================================                                   
The following groups are no longer being used by spamd-20060330nb10,                                          
and they can be removed if no other software is using them:                                                   
                                                                                                              
        _spamd                                                                                                
                                                                                                              
===========================================================================                                   
===========================================================================                                   
The following files are no longer being used by spamd-20060330nb10,                                           
and they can be removed if no other packages are using them:                                                  
                                                                                                              
        /usr/pkg/etc/spamd.conf                                                                               
                                                                                                              
===========================================================================                                   
searching for changes                                                                                         
remote: adding changesets                                                                                     
remote: adding manifests                                                                                      
remote: adding file changes                                                                                   
remote: added 5 changesets with 6 changes to 3 files                                                          
pushing to ssh://vcs@192.168.100.112/usr/home/vcs/pkgconftest                                                 
searching for changes                                                                                         
no changes found                                                                                              
pulling from ssh://vcs@192.168.100.112/usr/home/vcs/pkgconftest                                               
searching for changes                                                                                         
no changes found                                                                                              
defaults/usr/pkg/etc/spamd.conf already tracked!                                                              
REGISTER /var/hglocaldir/defaults//usr/pkg/etc/spamd.conf                                                     
spamd-20060330nb10: /usr/pkg/etc/spamd.conf already exists                                                    
spamd-20060330nb10: attempting to merge /usr/pkg/etc/spamd.conf with new defaults!                            
Saving the currently installed revision to /var/hglocaldir/automerged//usr/pkg/etc/spamd.conf                 
automerged/usr/pkg/etc/spamd.conf already tracked!                                                            
Failed to commit conf: backup preexisting conf before attempting merge for spamd-20060330nb10                 
pushing to ssh://vcs@192.168.100.112/usr/home/vcs/pkgconftest                                                 
searching for changes                                                                                         
no changes found                                                                                              
hg: failed to push changes to the remote repository ssh://vcs@192.168.100.112/usr/home/vcs/pkgconftest        
Merged with no conflict. installing it to /usr/pkg/etc/spamd.conf!                                            
Revert from the penultimate revision of /var/hglocaldir/automerged//usr/pkg/etc/spamd.conf if needed          
Failed to commit conf: add spamd-20060330nb10                                                                 
pushing to ssh://vcs@192.168.100.112/usr/home/vcs/pkgconftest                                                 
searching for changes                                                                                         
no changes found                                                                                              
hg: failed to push changes to the remote repository ssh://vcs@192.168.100.112/usr/home/vcs/pkgconftest        
===========================================================================                                   
The following files should be created for spamd-20060330nb10:                                                 
                                                                                                              
        /etc/rc.d/pfspamd (m=0755)                                                                            
            [/usr/pkg/share/examples/rc.d/pfspamd]                                                            
[...]

Yes, hg exits in error when there are no changes to be pushed. Everything is working fine.

Let's simulate some more package-provided changes in defaults

pkghost# make clean; sed -i "s/PKGREVISION=10/PKGREVISION=11/g" Makefile; make extract
pkghost# sed -i "s/http/https/g" work/spamd-20060330/etc/spamd.conf 
pkghost# vi work/spamd-20060330/etc/spamd.conf; head work/spamd-20060330/etc/spamd.conf
work/spamd-20060330/etc/spamd.conf: 87 lines, 2812 characters                                                 
                                                             .                                                
# $OpenBSD: spamd.conf,v 1.17 2006/02/01 20:22:43 dhartmei Exp $                                              
#                                                                                                             
# spamd config file, read by spamd-setup(8) for spamd(8)                                                      
#                                                                                                             
# See spamd.conf(5)                                                                                           
#THIS IS A NEW COMMENT!                                                                                       
#                                                                                                             
# Configures whitelists and blacklists for spamd                                                              
#                                                                                                             
# Strings follow getcap(3) convention escapes, other than you                                                 
pkghost# make update                                                                                          
===> Skipping vulnerability checks.                                                                           
WARNING: No /var/db/pkg/pkg-vulnerabilities file found.                                                       
WARNING: To fix run: `/usr/pkg/sbin/pkg_admin -K /var/db/pkg fetch-pkg-vulnerabilities'.                      
===> Deinstalling for spamd-20060330nb11                                                                      
Running /usr/pkg/sbin/pkg_delete -K /var/db/pkg -r spamd-20060330nb10                                         
===========================================================================                                   
The following users are no longer being used by spamd-20060330nb10,                                           
and they can be removed if no other software is using them:  

[...]

===> Installing for spamd-20060330nb11                                                                        
[...]
=> Automatic manual page handling                                                                             
=> Generating post-install file lists                                                                         
=> Checking file-check results for spamd-20060330nb11                                                         
=> Creating binary package /root/pkgsrc/mail/spamd/work/.packages/spamd-20060330nb11.tgz                      
===> Building binary package for spamd-20060330nb11                                                           
=> Creating binary package /root/pkgsrc/packages/All/spamd-20060330nb11.tgz                                   
===> Installing binary package of spamd-20060330nb11                                                          
abort: repository usr/home/vcs/pkgconftest already exists!                                                    
abort: could not create remote repo!                                                                          
pushing to ssh://vcs@192.168.100.112/usr/home/vcs/pkgconftest                                                 
searching for changes                                                                                         
no changes found                                                                                              
pulling from ssh://vcs@192.168.100.112/usr/home/vcs/pkgconftest                                               
searching for changes                                                                                         
no changes found                                                                                              
defaults/usr/pkg/etc/spamd.conf already tracked!                                                              
REGISTER /var/hglocaldir/defaults//usr/pkg/etc/spamd.conf                                                     
spamd-20060330nb11: /usr/pkg/etc/spamd.conf already exists                                                    
spamd-20060330nb11: attempting to merge /usr/pkg/etc/spamd.conf with new defaults!                            
Saving the currently installed revision to /var/hglocaldir/automerged//usr/pkg/etc/spamd.conf                 
automerged/usr/pkg/etc/spamd.conf already tracked!                                                            
Conf commit: pkgsrc: backup preexisting conf before attempting merge for spamd-20060330nb11                   
pushing to ssh://vcs@192.168.100.112/usr/home/vcs/pkgconftest                                                 
searching for changes                                                                                         
remote: adding changesets                                                                                     
remote: adding manifests                                                                                      
remote: adding file changes                                                                                   
remote: added 1 changesets with 1 changes to 1 files                                                          
Merged with no conflict. installing it to /usr/pkg/etc/spamd.conf!                                            
--- /usr/pkg/etc/spamd.conf     2018-08-04 10:43:37.554182167 +0000                                           
+++ /var/hglocaldir/defaults//usr/pkg/etc/spamd.conf.automerge  2018-08-04 10:50:23.582806647 +0000           
@@ -3,6 +3,7 @@                                                                                               
 # spamd config file, read by spamd-setup(8) for spamd(8)                                                     
 #                                                                                                            
 # See spamd.conf(5)                                                                                          
+#THIS IS A NEW COMMENT!                                                                                      
 #                                                                                                            
 # Configures whitelists and blacklists for spamd                                                             
 #                                                                                                            
Revert from the penultimate revision of /var/hglocaldir/automerged//usr/pkg/etc/spamd.conf if needed          
Conf commit: pkgsrc: add spamd-20060330nb11                                                                   
pushing to ssh://vcs@192.168.100.112/usr/home/vcs/pkgconftest                                                 
searching for changes                                                                                         
remote: adding changesets                                                                                     
remote: adding manifests                                                                                      
remote: adding file changes                                                                                   
remote: added 1 changesets with 2 changes to 2 files                                                          
===========================================================================                                   
The following files should be created for spamd-20060330nb11:                                                 
[...]

Also see GSoC 2018 Reports: Configuration files versioning in pkgsrc, part 4: configuration deployment, pkgtools and possible improvements

[0 comments]

 



Post a Comment:
  • HTML Syntax: NOT allowed