ZFS Native Encryption on Linux: It's Here Now! Kinda!

ZFS is getting native encryption. I'm pumped. It's a game changer.

ZFS almost had native encryption back when Sun Microsystems still existed, but the implementing team was apparently not packed with A-listers and the feature was scrapped. Sun died out before they could take a second swing at it, so over the years we've had to settle for compromises like GELI and LUKS to handle our full disk encryption needs. And it was good.

But it's going to get better.

This is a brave new world and we are all going to have to explore it.

The framework for native encryption was added back in the v0.7.x branch, but full-on, real-deal encryption is coming in v0.8.0 which ships Real Soon Now. The ZFS on Linux project is up to 0.8.0 release candidate 5, so we're really, really close to ZFS encryption Nirvana.

But we're not there yet.

I was lamenting this fact last month while sitting through a ZFS talk that Allan Jude was giving at LinuxFest Northwest. I was thinking that if ZFS on Linux is going to be the first open source ZFS implementation with native encryption, if it ships tomorrow, it will still be months before that magic release gets adopted into Debian/Ubuntu/Mint and becomes an out-of-the-box feature I can use. And I became depressed.

So one of the things that has constantly vexed me about using ZFS on Debian-based distros is that, other than Ubuntu, ZFS needs to be added as a DKMS module and that makes upgrades a delicate matter to approach. I've covered how to handle this in the past.

Impatiently, I set out to look into how to get the raw ZFS on Linux source to compile, without the middle man. And, surprisingly, it's doable. It's even more fragile than using DKMS, but it works. So while at some point the major Linux distros and FreeBSD will have native ZFS encryption that you can use right away when you set out to install your OS, you can get started, carefully, today. And that's what I'm going to cover in this howto.

To start with the quick version, this is NOT going to cover how to deploy native encryption with a ZFS on root Linux setup. We'll get there, in time. Every baby learns to crawl before it learns to walk.

Instead, we are going to end up creating a portable set of .deb package files we can install on any suitable machine if it matches our exact kernel version and architecture. These instructions are largely taken from https://www.klaus-hartnegg.de/gpo/2017-11-29-ZFS-in-Devuan.html but these instructions are slightly out-of-date, so consider this an addendum to that page.

1. First things first, we download an install ISO of Devuan Linux and boot it. I use the file "devuan_ascii_2.0.0_amd64_minimal-live.iso" either copied to a USB drive with Rufus or Etcher, or attached to a virtual DVD drive when creating a VM. We are going to install a regular ol' Devuan distro on a regular uncool file system like ext2. This is going to be our ZFS source compiler machine, and it can easily be done in a VM so long as you give it at least 4GB of RAM. I also recommend 4 cores on your CPU, but you can get by with fewer if you're patient. This howto assumes an amd64 architecture. Your actual mileage may vary.
   
   
2. Install Devuan. Basically this involves: partitioning your disk, formatting the new partition, mounting the new partition, running debootstrap in order to download and extract core OS packages to the new partition, mounting several mountpoints (/dev, /dev/pts, /proc, and /sys) inside the new partition, setting a root password, setting timezone and locale data, configuring the network, and setting the bootloader. This seems like a lot, but we've done this a bunch of times on this blog and it should start to feel pretty routine by now.
   
   

   If you really want a rundown of what to run in the Devuan minimal ISO to install Devuan, it'd look something like this:
   
   

   DEVICE=/dev/sda
   PARTITIONNUMBER=1
   TARGET=/mnt
   ARCH=amd64
   BRANCH=ascii
   MIRROR=https://pkgmaster.devuan.org/merged
   PKGS=console-setup,kbd,locales,tmux,openssh-client
   KEYRINGDIR=/usr/share/keyrings
   
   dd if=/dev/zero of=${DEVICE} bs=1M count=2
   /sbin/parted --script --align opt ${DEVICE} mklabel msdos
   /sbin/parted --script --align opt ${DEVICE} mkpart pri 1MiB 100%
   /sbin/parted --script --align opt ${DEVICE} set ${PARTITIONNUMBER} boot on
   
   mkfs.ext2 ${DEVICE}${PARTITIONNUMBER}
   mount ${DEVICE}${PARTITIONNUMBER} ${TARGET}
   
   dhclient eth0
   
   /usr/sbin/debootstrap \
     --arch=${ARCH} \
     --include=${PKGS} \
     ${BRANCH} \
     ${TARGET} \
     ${MIRROR}
   
   cp -v -r -p ${KEYRINGDIR} ${TARGET}/usr/share/
   
   mkdir -p ${TARGET}/etc/apt/sources.list.d
   mkdir -p ${TARGET}/usr/share/keymaps
   mkdir -p ${TARGET}/etc/network
   
   # Add eth0 DHCP config to /etc/network/interfaces
   cp -p ${TARGET}/etc/network/interfaces ${TARGET}/etc/network/interfaces.bak
   cp -p ${TARGET}/etc/network/interfaces ${TARGET}/etc/network/interfaces.new
   echo "auto eth0" >> ${TARGET}/etc/network/interfaces.new
   echo "iface eth0 inet dhcp" >> ${TARGET}/etc/network/interfaces.new
   chmod 0644 ${TARGET}/etc/network/interfaces.new
   mv -v -f ${TARGET}/etc/network/interfaces.new ${TARGET}/etc/network/interfaces
   
   cp -p ${TARGET}/etc/apt/sources.list ${TARGET}/etc/apt/ # this will get ascii-security too
   
   install -m0644 /etc/hostname ${TARGET}/etc/
   echo 'en_US.UTF-8 UTF-8' > ${TARGET}/etc/locale.gen
   ln -sf /proc/self/mounts ${TARGET}/etc/mtab
   
   cat /etc/resolv.conf > ${TARGET}/etc/resolv.conf.new
   chmod 0644 ${TARGET}/etc/resolv.conf.new
   mv -v -f ${TARGET}/etc/resolv.conf.new ${TARGET}/etc/resolv.conf
   
   for i in /dev /dev/pts /proc /sys
   do
     echo -n "mount $i..."
     mount -B $i ${TARGET}$i
     echo 'done!'
   done
   
   chroot /mnt env DEBIAN_FRONTEND=noninteractive dpkg-reconfigure locales
   chroot /mnt env DEBIAN_FRONTEND=noninteractive dpkg-reconfigure tzdata
   chroot /mnt apt-get update
   chroot /mnt apt-get install -y linux-image-${ARCH}
   chroot /mnt env DEBIAN_FRONTEND=noninteractive apt-get install -y grub-pc
   chroot /mnt passwd -u root
   chroot /mnt passwd root
   < enter a root password >
   chroot /mnt update-initramfs -u -k all
   chroot /mnt update-grub
   chroot /mnt grub-install ${DEVICE}
   
   for i in sys proc dev/pts dev
   do
     umount ${TARGET}/$i
   done
   
   halt -p

   
   
3. Remove the live CD and restart the machine. This should give you a working Devuan install, albeit a pretty sparse one. In order to compile the ZFS on Linux source on it, you'll need to login and begin the real setup:
   
   

   apt-get update
   apt-get install -y \
     alien autoconf build-essential dirmngr fakeroot gawk \
     gnupg2 ksh libattr1-dev libblkid-dev libselinux1-dev \
     libssl-dev libtool libudev-dev linux-headers-$(uname -r) \
     lsscsi parted python3 python3-dev python3-pip \
     uuid-dev zlib1g-dev
   
   pip3 install setuptools
   pip3 install cffi

   
   
4. Don't compile things as root. Make a new user account to use for the rest of the procedure:
   
   

   groupadd source
   useradd -g source -d /home/source -s /bin/bash -m source
   su -l source

   
   

   Download ZFS v0.8.0. (Hard to do because at the time of this writing it doesn't exist yet. I'm using the rc5 release:
   
   

   https://github.com/zfsonlinux/zfs/releases/download/zfs-0.8.0-rc5/zfs-0.8.0-rc5.tar.gz
   
   https://github.com/zfsonlinux/zfs/releases/download/zfs-0.8.0-rc5/zfs-0.8.0-rc5.tar.gz.asc
   
   

   These will soon be obsoleted, but ya gotta start somewhere.
   
   

5. Verify the tar.gz you downloaded has been signed by the project's signing key.
   
   

   /usr/bin/gpg2 --verbose --keyserver keys.gnupg.net --recv-key 0AB9E991C6AF658B
   /usr/bin/gpg2 --verbose --verify zfs-0.8.0-rc5.tar.gz.asc

   
   
6. If the signature is good, extract the tarball.
   
   

   SHORTVERSION=0.8.0
   LONGVERSION=${SHORTVERSION}-rc5
   
   gzip -d < ./zfs-${LONGVERSION}.tar.gz | tar -xf -
   cd ~source/zfs-${SHORTVERSION}
   sh ./autogen.sh
   ./configure
   make -j $(nproc)
   make deb
   

   
   
7. This will create number of .deb package files you should relocate to a safe location, like another machine or a USB thumb drive, or both. To install ZFS on this machine, or any other, install the .deb files manually with dpkg as root:
   
   

   ARCH=amd64
   KERNELVERSION=$(uname -r)
   SHORTVERSION=0.8.0
   LONGVERSION=${SHORTVERSION}-0
   
   cd ~source/zfs-${SHORTVERSION}
   dpkg -i zfs_${LONGVERSION}_${ARCH}.deb
   dpkg -i kmod-zfs-${KERNELVERSION}_${LONGVERSION}_${ARCH}.deb
   dpkg -i libnvpair1_${LONGVERSION}_${ARCH}.deb
   dpkg -i libuutil1_${LONGVERSION}_${ARCH}.deb
   dpkg -i libzpool2_${LONGVERSION}_${ARCH}.deb
   dpkg -i libzfs2_${LONGVERSION}_${ARCH}.deb

   
   
8. To use ZFS, make sure your kernel modules are loaded:
   
   

   modprobe zfs

From here, you can begin creating zpools and zfs datasets on this box, which may or may not be useful to you if you are hellbent on a ZFS-on-root setup, but this is enough for you to start working with the encryption feature of ZFS v0.8.0 as a learning tool.

Note: I only very briefly touched on ascii-security. This apt-get branch is useful for getting old versions of the kernel and kernel headers installed. Since the ascii install ISO uses a slightly outdated kernel version, 4.9.0-6, if you specify installing this exact kernel on your source compiler machine, you can then re-use the .deb files it creates in conjunction with the install ISO to create a ZFS-on-root Devuan machine in a very similar manner as we described in the first steps here. In other words, making this change:

- chroot /mnt apt-get install -y linux-image-${ARCH}
+ chroot /mnt apt-get install -y linux-image-$(uname -r)

during the initial setup of your compiling machine will put the same kernel on that machine as the ascii ISO live CD uses. Thus the .deb files you build on it can be installed to the live CD environment with dpkg to create a zpool. You can even do this on the same hardware once you've built the .deb files, at the cost of having to install Devuan twice, and repeat the process whenever you choose to change the kernel.

Note also that while the ZFS compiling process creates a zfs-initramfs .deb, we don't install it. For a ZFS-on-root scenario, you'd want to make sure that it's included as well or else your machine will be unbootable.