VPSBuildStandard
LiveCD Environment
- Boot a Ubuntu Server ISO
- From the
[ Help ]
menu, selectEnter Shell
passwd
echo "PermitRootLogin yes" >/etc/ssh/sshd_config.d/root.conf
systemctl restart sshd
- Connect to the VPS with SSH
wget -qO- https://cloud-images.ubuntu.com/jammy/current/jammy-server-cloudimg-arm64-root.tar.xz | tar -xJf -
Linux OS Partitions
export DEV=/dev/vda
blkdiscard "${DEV}" -f
sudo parted "${DEV}"
mklabel gpt
mkpart bios_grub ext2 2048s 8M
set 1 bios_grub on
mkpart ESP fat32 8M 512M
set 2 ESP on
mkpart BOOT ext2 512M 2G
mkpart SWAP linux-swap 2G 8G
mkpart SSD1 ext2 8G 100%
q
mkfs.fat -F 32 "${DEV}2"
ZFS Datasets
bpool
Possible improvement: Using "copies=2" makes it so that all data in a certain dataset is stored twice. This allows ZFS to automatically recover from corrupted blocks. This uses twice the space and, because both copies are on the same physical device, does not help in the event of device failure. This might be useful on bpool/BOOT/ROOT and SSD1/OS/Neon/ROOT.
The "bpool" dataset needs to be compatible with the GRUB bootloader so it needs to be created with a minimal subset of ZFS features. Why not just use ext4 or even the existing FAT32 EFI partition? ZFS still has checksuming (so you will know if your kernel image/initrd have become corrupted) and snapshots (for rollback).
sudo apt install -y zfsutils-linux
sudo zpool create -d \
-o feature@async_destroy=enabled \
-o feature@empty_bpobj=enabled \
-o feature@spacemap_histogram=enabled \
-o feature@enabled_txg=enabled \
-o feature@hole_birth=enabled \
-o feature@bookmarks=enabled \
-o feature@embedded_data=enabled \
-o feature@large_blocks=enabled \
-O mountpoint=/mnt/zfs/bpool \
-f \
bpool /dev/disk/by-partlabel/BOOT
sudo zfs create bpool/BOOT
sudo zfs create bpool/BOOT/ROOT
sudo zfs set mountpoint=/boot bpool/BOOT/ROOT
SSD1
TODO: use zstd instead of lz4?
sudo zpool create -o ashift=12 \
-o feature@async_destroy=enabled \
-o feature@encryption=enabled \
-o feature@bookmarks=enabled \
-o feature@embedded_data=enabled \
-o feature@empty_bpobj=enabled \
-o feature@enabled_txg=enabled \
-o feature@extensible_dataset=enabled \
-o feature@filesystem_limits=enabled \
-o feature@hole_birth=enabled \
-o feature@large_blocks=enabled \
-o feature@lz4_compress=enabled \
-o feature@spacemap_histogram=enabled \
-o feature@userobj_accounting=enabled \
-O acltype=posixacl \
-O compression=lz4 \
-O devices=off \
-O normalization=formD \
-O relatime=on \
-O xattr=sa \
-O mountpoint=/mnt/zfs/SSD1 \
-f \
SSD1 /dev/disk/by-partlabel/SSD1
sudo zfs create SSD1/OS
sudo zfs create -o encryption=aes-256-gcm -o keyformat=passphrase -o compression=on SSD1/OS/Jammy
sudo zfs load-key SSD1/OS/Jammy
sudo zfs create SSD1/OS/Jammy/ROOT
Copy the upstream OS RootFS to the ZFS datasets
sudo zpool export SSD1
sudo zpool export bpool
sudo mkdir /target
sudo zpool import -R /target SSD1
sudo zpool import -R /target bpool
sudo zfs set overlay=on SSD1/OS/Jammy/ROOT
sudo zfs set mountpoint=/ SSD1/OS/Jammy/ROOT
sudo zpool export bpool
sudo zpool export SSD1
sudo zpool import -R /target SSD1
sudo zfs load-key SSD1/OS/Jammy
sudo zfs mount SSD1/OS/Jammy/ROOT
sudo zpool import -R /target bpool
mkdir /target/boot/efi
sudo mount /dev/disk/by-partlabel/ESP /target/boot/efi
wget -qO- https://cloud-images.ubuntu.com/jammy/current/jammy-server-cloudimg-amd64-root.tar.xz | tar -xJf - -C /target
Make the OS in the ZFS datasets bootable
We bind certain system directories into the /target directory and then chroot into it.
sudo mkdir /target/dev /target/proc /target/sys
sudo mount --rbind /dev /target/dev
sudo mount --rbind /proc /target/proc
sudo mount --rbind /sys /target/sys
sudo mount --rbind /dev/pts /target/dev/pts
sudo mount --make-rslave /target/dev
sudo mount --make-rslave /target/proc
sudo mount --make-rslave /target/sys
sudo mount --make-rslave /target/dev/pts
sudo chroot /target
export DEV=/dev/vda
unlink /etc/resolv.conf
echo "nameserver 8.8.8.8" >/etc/resolv.conf
Not relevant yet: If your hardware needs a kernel newer than the one provided (5.15) you can install the HWE kernel (X.X) now:
apt install linux-image-generic-hwe-XX.XX linux-headers-generic-hwe-XX.XX
Old version
sudo apt install zfsutils-linux zfs-initramfs openssh-server linux-image-generic
sudo apt purge snapd
rm -fr /etc/default/grub.d/50-cloudimg-settings.cfg
useradd ubuntu
passwd ubuntu
usermod -a -G ubuntu sudo
echo "ChallengeResponseAuthentication yes" >/etc/ssh/sshd_config.d/ChallengeResponseAuthentication.conf
hostnamectl set-hostname lu.ghanima.net
echo "127.0.0.1 localhost lu lu.ghanima.net" >>/etc/hosts
echo -e "network:\n ethernets:\n enp1s0:\n dhcp4: true\n version: 2" >/etc/netplan/01-manual-configuration.yaml
echo "options zfs zfs_arc_max=134217728" >> /etc/modprobe.d/zfs.conf
sudo update-initramfs -c -k all
New version
apt update
apt install -y zfsutils-linux zfs-initramfs openssh-server linux-image-generic
rm -fr /etc/default/grub.d/50-cloudimg-settings.cfg
echo -e "network:\n ethernets:\n enp1s0:\n dhcp4: true\n version: 2" >/etc/netplan/01-manual-configuration.yaml
echo "GRUB_CMDLINE_LINUX_DEFAULT=\"'ds=nocloud-net;s=https://configuration-backups.ghanima.net/cloud-init/lu.ghanima.net/'\"" >/etc/default/grub.d/cloud-init.cfg
echo "GRUB_TIMEOUT=5" >>/etc/default/grub.d/cloud-init.cfg
sudo update-initramfs -c -k all
The following error messages are normal (we are not using dmcrypt so they are harmless):
cryptsetup: ERROR: Couldn't resolve device SSD1/OS/Jammy/ROOT
cryptsetup: WARNING: Couldn't determine root device
sudo grub-install --bootloader-id=ubuntu --efi-directory=/boot/efi "${DEV}"
Setup encrypted swap
echo "swap /dev/disk/by-partlabel/SWAP /dev/urandom swap,cipher=aes-cbc-essiv:sha256,size=256,plain" | sudo tee -a /etc/crypttab
echo "/dev/mapper/swap none swap defaults 0 0" | sudo tee -a /etc/fstab
echo "RESUME=none" |sudo tee /etc/initramfs-tools/conf.d/resume
Update everything
apt update; apt -y full-upgrade; apt autoremove
sudo zfs snapshot SSD1/OS/Jammy/ROOT@fresh-install
sudo zfs snapshot bpool/BOOT/ROOT@fresh-install
sudo update-grub
exit
sudo umount /target/boot/efi
sudo umount -R /target
sudo zfs umount SSD1/OS/Jammy/ROOT
sudo zpool export bpool
sudo zpool export SSD1
The following errors can be ignored.
umount: /target/mnt/zfs/SSD1/OS: no mount point specified.
cannot unmount '/target/mnt/zfs/SSD1/OS': umount failed
Now we reboot, hopefully into our newly installed OS.
sudo reboot
You should be prompted to unplug the USB boot media during shutdown. If not, unplug it during BIOS startup, before the machine boots.