1 PXE install w multi-boot, btrfs & Libreboot support
3 Some things are specific to my home network, and uses files with secrets
4 that are not in this repo. I use this for bare metal and vms, and two
5 scripts which can run post boot so I use them on vps distributed image
8 Features people may find useful: installs encrypted trisquel, debian,
9 ubuntu, arch, and parabola (archlike install is likely broken, I've only
10 done pxe boots recently), in a multi-boot setup using multiple
11 subvolumes of a single btrfs filesystem. Utilizes multiple disks, with
12 scripts to automatically decrypt on intentional reboots, but not after
13 shutdown or power loss.
15 Normal install mode for fai is using pxe, but on a libreboot system,
16 there is no pxe. The pxe in a normal computer is nonfree
17 firmware. Alternatives to normal pxe that I've tried:
19 * libreboot + seabios + ipxe
21 * Use a live cd to call pxe-kexec, this is described later in this file.
23 * Use the fai autodiscover iso. This is more automated, so nicer.
25 * Use an install method above to setup a gnu/linux disk partition that
26 coordinates with libreboot grub to acts like a pxe boot using
27 kexec. The boot process takes a bit longer than normal pxe. This is
28 the bootstrap partition in my scripts.
30 Things I haven't tried:
32 * The bios chip has enough room for an initrd. This could be setup to
33 work like the partition I use to kexec, but it would be faster, and
34 not require installing to disk.
36 The partititioning and filesystem script is at
37 fai/config/hooks/partition.DEFAULT. Disks are grouped as ssd or hdd and
38 raided in raid 1 or raid 0 per configuration. The base partitions are
39 divided into boot, swap, and root, (only boot is unencrypted). There are
40 scripts to resize those partitions post-provision and while the system
43 People who use fai may find these things as useful examples: it uses
44 dnsmasq (on a openwrt machine) for dhcp instead of the isc
45 dhcp. fai-wrapper is a small script to use basic fai classes outside of
46 fai. It does not use the fai partitioning tool, but the script is
47 inspired from it and works outside of fai. It supports running a fai
48 server on debian within android via Maru.
50 It also automates configuration of an openwrt router after manual
53 After provisionining is done, I sync files using btrfs, or unison for
54 vps, then automate further setup using a different set of scripts,
55 https://iankelling.org/git/?p=distro-setup;a=tree.
57 My network is a wndr3700v2 router with openwrt on it and a few pcs/laptops.
59 Since fai requires a debian server as the fai server, there are also
60 scripts to automate a debian install using pxe and preseeding, which can
61 be done from any distro.
63 Some of the scripts have dependencies for some simple obvious utility
64 scripts from https://iankelling.org/git, and of course there are some
65 hostnames that are specific to my network.
68 # Per-host/install configuration
70 Before doing a fai install, you will need to populate a class file. I
71 use one called 51-multi-boot, which you can see example of in
72 fai/config/class/50-host-classes.
76 Before doing a fai install, you will need to populate /q/root/luks and
77 /q/root/shadow, see their references. You might also want to copy
78 existing /etc/ssh/*host* to
79 /p/c/machine_specific/HOST/filesystem/etc/ssh.
81 host-* luks keyfiles generated like:
82 head -c 2048 /dev/urandom | od | s dd of=/q/root/luks/host-demohost
84 Configuration of which luks key to use is in
85 fai/config/hooks/partition.DEFAULT
87 Configuration of which (if any) shadow file to use is in
88 fai/config/distro-install-common/end
89 and which shadow file / luks file(s) to copy into the new machine depends
90 on fai-redep arguments.
92 # Scripts (meant to be used directly):
95 # Setup the environment for the install
97 # create tiny autodiscover cd
98 # todo: with fai-revm at least, this complains about missing vmlinuz. need to fix this.
99 fai-redep && sudo fai-cd -g $PWD/grub.cfg.autodiscover -f -A $BASEFILE_DIR/autodiscover.iso
100 # create normal fai cd (replace TARGET_HOSTNAME)
101 fai-redep -t TARGET_HOSTNAME && sudo fai-cd -M -g $PWD/grub.cfg.netinst-noreboot -f $BASEFILE_DIR/netinst.iso
102 # note, may need to set hostname, depending on config,
103 # and some other things for environment not on your lan
104 # for example see fai/config/class/LINODE.var. See linode notes below.
106 mymk-basefile # Create basefiles for various distros
107 archlike-pxe # Setup pxe boot server from an archlike base image
108 fai-redep # Deploy fai configuration to host "faiserver"
109 faiserver-uninstall # uninstall fai-server
110 faiserver-setup # install fai-server on the current machine
111 myfai-chboot # setup fai tftp and nfs. useful for doing pxe-kexec
112 pxe-server # disable/enable pxe dhcp, tfp, and nfs. calls myfai-chboot
113 wrt-setup # setup my router in general: dhcp, dns, etc.
116 # Script to do a distro install
118 faiserver-revm # using pxe & preseed, create a vm which is a fai server
119 dsfull # install & post-install a new fai distro
120 arch-init-remote # install arch after it's been booted into it's setup env
121 live-kexec # Kexec this or a remote machine using host faiserver. also
122 useful to run as curl live-kexec|bash
127 arch-revm # test arch install on a fresh vm
128 fai-revm # test fai install on a fresh vm
131 # Scripts to call after a distro install for various reasons
133 chboot # Set grub to boot into a different distro (installed earlier)
134 install-chboot # reinstall chboot to /boot subvols, for chboot updates.
135 eboot # reboot without automatic disk decryption
136 fai-wrapper # use fai classes outside of fai. sourced, not called.
137 faiserver-disable # Disable the fai nfs server exports
138 fresize # resize swap or boot partitions in a host
141 # Replacing a raid 10 disk
143 # i expect better results with newer kernel and btrfs progs than the default stretch
146 pxe-server -S HOST fai
148 # btrfs replace or delete. prefer replace. to setup partitions on replacement drive:
149 scp fai-wrapper HOST:
152 export SPECIAL_DISK=/dev/REPLACEMENT_DEV
153 /var/lib/fai/config/hooks/partition.DEFAULT
157 for x in /target/* /target; do umount $x; done
159 PASSWORD HERE(ctrl-d ctrl-d)
161 for d in ata*part1; do cryptsetup luksOpen -d /root/p $d crypt_dev_$d; done
162 x=(/dev/mapper/*part1); mount -o subvol=root_trisquelflidas $x /mnt
164 # btrfs replace start -f /dev/mapper/OLD_DEV /dev/mapper/NEW_DEV /mnt
165 # btrfs replace status /mnt
166 # nohup btrfs dev delete /dev/sde1 /mnt
167 mount -o subvol=boot_trisquelflidas /dev/sda3 /mnt/boot
168 # also replace or delete disk for boot
169 for x in dev proc sys; do mount -o bind /$x /mnt/$x; done
170 chroot /mnt /bin/bash
171 # replace disk in fstab
172 # replace disk in /etc/crypttab
181 # Expected output in fai logs
183 For flidas, when installing systemd, this error happens, and it's
184 a superflous upstream bug based on reading the post install script:
186 addgroup: The group `systemd-journal' already exists as a system group. Exiting.
187 Operation failed: No such file or directory
192 * create 2 disks, installer (3000 mb, raw), boot (remaining, raw)
193 * create 2 profiles w direct boot, no helpers:
194 * installer (sda=boot, sdb=installer, boot dev=sdb)
196 * Boot into rescue mode, ssh in with lish,
197 curl url_to_some_fai_cd_created_image | dd of=/dev/sda
199 * boot into installer.
200 * Lish shows console, at the end of install, it gives prompt because
201 logs failed to save remotely, check the logs, then reboot into boot
202 profile if all is well. If that doesn't happen, turn off lassie in
208 For someone who really needed ubuntu on host tp, otherwise they would
209 end up on a non-gnu os, and I didn't want to figure out how to get all
210 the default software installed, I did the following:
215 # set 51-multi-boot to set classes outside of fai-wrapper conditional, including NOWIPE
217 ./fai/config/hooks/partition.DEFAULT
220 # install ubuntu 20.04 using virt-install
222 virt-install --os-variant=ubuntu16.04 --cdrom ubuntu-20.04-desktop-amd64.iso --disk path=u2004.qcow2 -r 2048 --vcpus 1 -n u2004
223 qemu-img create -o preallocation=metadata -f qcow2 u2004.qcow2 15G
224 # alternatively, also tried a physical install, because I know the virtual install ends up
225 # with some differen things, like some spice service. then pulled the data out with
226 rsync -ahSAX --numeric-ids --exclude=proc --exclude=sys --exclude=dev --exclude=tmp --exclude=run root@tp:/ .; mkdir proc sys dev tmp
229 qemu-nbd --connect=/dev/nbd0 u1804.qcow2 -f qcow2
230 qemu-nbd --connect=/dev/nbd0 u2004.qcow2 -f qcow2
231 mount /dev/nbd0p1 /mnt/1 # bionic
232 mount /dev/nbd0p5 /mnt/1 # focal
233 mount -o bind /mnt/root/root_ubuntubionic /mnt/2
234 mount -o bind /mnt/root/root_ubuntufocal /mnt/2
236 mount -o bind /mnt/boot/boot_ubuntubionic /mnt/2/boot
237 mount -o bind /mnt/boot/boot_ubuntufocal /mnt/2/boot
238 # S = sparse, A = acls, X = xattrs
239 rsync -ahSAX --numeric-ids /mnt/1/ /mnt/2
242 cp /tmp/fai/crypttab etc
243 sed -i "s#/root/keyscript,#decrypt_keyctl,#" etc/crypttab
244 cp /tmp/fai/fstab etc
245 echo "tmpfs /tmp tmpfs nodev,nosuid,size=50%,mode=1777 0 0" >> etc/fstab
248 mv /etc/resolv.conf /etc/resolv.conf.old
249 echo nameserver 1.1.1.1 >/etc/resolv.conf
250 # install programs from /a/bin/fai/fai/config/package_config/STANDARD:
251 apt install -y openssh-client openssh-server cryptsetup keyutils btrfs-progs console-setup kbd pciutils usbutils unattended-upgrades initramfs-tools-core dropbear-initramfs
252 mv /etc/resolv.conf.old /etc/resolv.conf
254 d=etc/initramfs-tools
255 mkdir -p $d/root/.ssh etc/dropbear-initramfs root/.ssh
256 chmod 700 $d/root $d/root/.ssh root/.ssh
257 cp -p /root/.ssh/authorized_keys $d/root/.ssh/authorized_keys
258 cp -p /root/.ssh/authorized_keys etc/dropbear-initramfs
259 cp -p /root/.ssh/authorized_keys root/.ssh/authorized_keys
261 sed -ri 's/^ *GRUB_CMDLINE_LINUX_DEFAULT=.*/GRUB_CMDLINE_LINUX_DEFAULT="rd.luks.crypttab=no"/' /etc/default/grub
262 grub-install --no-floppy $(grub-probe -tdrive -d /dev/sda)
264 grub-bios-setup -d /boot/grub/i386-pc -s /dev/sda
272 # the only useful image is ubuntu 18.04 ayafun or something.
274 s mount /dev/sdb7 /mnt/1
275 s cp `which qemu-arm-static` /mnt/1/usr/bin
276 s chroot /mnt/1 qemu-arm-static /bin/bash
277 usermod --login iank --move-home --home /home/iank rock46
278 groupmod --new-name iank rock64
286 Change arch to archlike and to support arch and parabola
291 The license for the project is GPLv2 or later, mostly because fai is and
292 I periodically merge the upstream example config, which contains small
293 scripts. Also, there is a modified encrypt.upstream, which is from the
294 cryptsetup package in arch, which is under the same license.