Contents

[NetBSD] Using latest-pkgsrc built in home directory

On NetBSD, some packages built with /usr/pkgsrc/ or installed via pkgin are too old for me. It applies especially to modern/actively-developed softwares, such like clang or fish.

But using only the latest pkgsrc means living on the bleeding-edge. And they get to be buggy and builds get to be broken only when we are busy (you know, it’s called Murphy’s law).

So I use both:

  • System’s stable packages (/usr/pkg/), installed with /usr/pkg/bin/pkgin
  • Latest pkgsrc

Even when packages in latest pkgsrc are broken, we can use stable packages!

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
# system's stable packages
/usr/pkgsrc/          # (I don't use)
/usr/pkg/             # Stable packages installed by pkgin
/usr/sbin/pkg_add     # (I don't use)
/usr/pkg/bin/pkgin    # Install stable binary packages

# latest packages
# sudo is not needed
$HOME/pkgsrc/           # latest pkgsrc
$HOME/pkg/              # latest pkg (compiled from $HOME/pkgsrc/)
$HOME/work-bootstrap/   # working directory for bootstrap
$HOME/pkg/bin/bmake     # compiles $HOME/pkgsrc/

Requirements

  • Latest version of NetBSD (9.1)
  • git installed with system’s pkgsrc, pkg_add, or pkgin.
  • Basic knowledge of using git (git clean, git rebase, dealing with force-push (git reset --hard))

I tried on NetBSD 7.1 amd64, but /usr/lib/libstdc++.so.7.3 was too old for the latest pkgsrc.

Procedures below was tested only on NetBSD 9.1 amd64.

Bootstrap

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
cd ~/  # or anywhere you want to clone pkgsrc
git clone https://github.com/NetBSD/pkgsrc.git
cd pkgsrc/

unset PKG_PATH  # if set: ERROR: Please unset PKG_PATH before running bootstrap.
bootstrap/bootstrap --help
# takes 3-30 minutes
bootstrap/bootstrap --unprivileged --prefix=$HOME/pkg --workdir=$HOME/work-bootstrap
export PATH="$HOME/pkg/bin:$HOME/pkg/sbin:$PATH"
which -a bmake  # $HOME/pkg/bin/bmake

Make and install packages

1
2
3
4
5
6
export PATH="$HOME/pkg/bin:$HOME/pkg/sbin:$PATH"
cd "$HOME"/pkgsrc/
vi mk/defaults/mk.conf  # MAKE_JOBS= 4
cd "$HOME"/pkgsrc/devel/git/
bmake  # "-j2" will be ignored; only MAKE_JOBS is used
bmake install

Here’s my convinient shell script:

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
#!/bin/sh

[ "${1+defined}" = defined ] || \
  { echo usage: sh "$0" '<path to pkgsrc directory>'  >&2; exit 1; }
DIR_PKGSRC=$1

dirs="
$(: ---------- want as fast as possible ----------)
devel/tig/
devel/git-contrib/
devel/git-svn/
devel/git/
shells/fish/
sysutils/htop/
textproc/the_silver_searcher/
$(: ---------- others ----------)
$(: devel/gdb/)
$(: devel/bmake/)
$(: lang/gcc9/)
$(: misc/vimpager/)
$(: security/gnupg/)
$(: shells/zsh/)
archivers/gtar/
converters/nkf/
devel/autoconf/
devel/automake/
devel/binutils/
devel/bison/
devel/check/
devel/cloc/
devel/cmake/
devel/cscope/
devel/doxygen/
devel/exctags/
devel/gdb/
devel/gettext/
devel/gmake/
devel/googletest/
devel/jq/
devel/libelf/
devel/libevent/
devel/libffi/
devel/libtool/
devel/lldb
devel/lldb/
devel/m4/
devel/ncurses/
devel/ncursesw/
devel/pcre/
devel/pcre2/
devel/pkgconf/
devel/py-pip/
devel/readline/
devel/subversion/
editors/emacs/
editors/vim/
emulators/qemu/
graphics/graphviz/
lang/clang/
lang/ghc/
lang/go/
lang/llvm/
lang/nodejs/
lang/python37/
lang/ruby/
math/mpfr/
misc/tmux/
mk/check/
net/iperf/
net/iperf3/
net/rsync/
net/socat/
net/tcpdump/
net/wget/
net/wireshark/ $(: for tshark)
pkgtools/pkglint/
security/gnupg2/
security/mozilla-rootcerts/
security/pinentry/
shells/bash/
sysutils/coreutils/
sysutils/lsof/
textproc/flyspell/
textproc/gsed/
textproc/ispell/
textproc/libyaml/
www/curl/
"

# export PYTHON_VERSION_DEFAULT=37
export CURSES_DEFAULT=ncursesw  # for tig

for dir in $dirs; do
  # echo "^$dir-"
  # echo "^$(basename "$dir")-"
  # pkg_info | grep "^$(basename "$dir")-"
  # pkg_info | grep "^$(basename "$dir")-[0-9]"
  if pkg_info | grep -q "^$(basename "$dir")-[0-9]"; then
    printf "\n\e[32m%s already installed: %s\e[37m\n" "$(date)" "$dir"
    pkg_info | grep "^$(basename "$dir")-[0-9]"
    printf '\e[0m'
    continue
  fi
  printf "\n\n\n\e[36m%s __grep_me__ %s\e[0m\n" "$(date)" "$dir"
  cd "$DIR_PKGSRC"/"$dir" || continue
  bmake && bmake install
  printf "\e[36m%s __grep_me__ %s done\e[0m\n" "$(date)" "$dir"
done 2>&1 | tee -a "$DIR_PKGSRC"/build.log

1
2
3
4
vi build_pkgsrc.sh                # paste above
sh build_pkgsrc.sh $HOME/pkgsrc/  # takes ~10 hours...
sh build_pkgsrc.sh $HOME/pkgsrc/  # builds only failed packages
less -R $HOME/pkgsrc/build.log

TODO: img

1
2
3
4
$ du -h -d1
20G     ./pkgsrc  # can be reduced with git clean -fd
84M     ./work-wpkg
2.6G    ./pkg