From: Ian Kelling Date: Sat, 7 Oct 2017 23:10:13 +0000 (-0700) Subject: Merge branch 'upstream' X-Git-Url: https://iankelling.org/git/?p=automated-distro-installer;a=commitdiff_plain;h=da61a304a10de72288782a9d872769fe44576f50;hp=7024f2155d8d5e4754d5c1ce0ccf8352149f81cd Merge branch 'upstream' --- diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..5e08dd9 --- /dev/null +++ b/.gitignore @@ -0,0 +1 @@ +/fai/config/class/51-multi-boot diff --git a/COPYING b/COPYING new file mode 100644 index 0000000..d159169 --- /dev/null +++ b/COPYING @@ -0,0 +1,339 @@ + GNU GENERAL PUBLIC LICENSE + Version 2, June 1991 + + Copyright (C) 1989, 1991 Free Software Foundation, Inc., + 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + Everyone is permitted to copy and distribute verbatim copies + of this license document, but changing it is not allowed. + + Preamble + + The licenses for most software are designed to take away your +freedom to share and change it. By contrast, the GNU General Public +License is intended to guarantee your freedom to share and change free +software--to make sure the software is free for all its users. This +General Public License applies to most of the Free Software +Foundation's software and to any other program whose authors commit to +using it. (Some other Free Software Foundation software is covered by +the GNU Lesser General Public License instead.) You can apply it to +your programs, too. + + When we speak of free software, we are referring to freedom, not +price. Our General Public Licenses are designed to make sure that you +have the freedom to distribute copies of free software (and charge for +this service if you wish), that you receive source code or can get it +if you want it, that you can change the software or use pieces of it +in new free programs; and that you know you can do these things. + + To protect your rights, we need to make restrictions that forbid +anyone to deny you these rights or to ask you to surrender the rights. +These restrictions translate to certain responsibilities for you if you +distribute copies of the software, or if you modify it. + + For example, if you distribute copies of such a program, whether +gratis or for a fee, you must give the recipients all the rights that +you have. You must make sure that they, too, receive or can get the +source code. And you must show them these terms so they know their +rights. + + We protect your rights with two steps: (1) copyright the software, and +(2) offer you this license which gives you legal permission to copy, +distribute and/or modify the software. + + Also, for each author's protection and ours, we want to make certain +that everyone understands that there is no warranty for this free +software. If the software is modified by someone else and passed on, we +want its recipients to know that what they have is not the original, so +that any problems introduced by others will not reflect on the original +authors' reputations. + + Finally, any free program is threatened constantly by software +patents. We wish to avoid the danger that redistributors of a free +program will individually obtain patent licenses, in effect making the +program proprietary. To prevent this, we have made it clear that any +patent must be licensed for everyone's free use or not licensed at all. + + The precise terms and conditions for copying, distribution and +modification follow. + + GNU GENERAL PUBLIC LICENSE + TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION + + 0. This License applies to any program or other work which contains +a notice placed by the copyright holder saying it may be distributed +under the terms of this General Public License. The "Program", below, +refers to any such program or work, and a "work based on the Program" +means either the Program or any derivative work under copyright law: +that is to say, a work containing the Program or a portion of it, +either verbatim or with modifications and/or translated into another +language. (Hereinafter, translation is included without limitation in +the term "modification".) Each licensee is addressed as "you". + +Activities other than copying, distribution and modification are not +covered by this License; they are outside its scope. The act of +running the Program is not restricted, and the output from the Program +is covered only if its contents constitute a work based on the +Program (independent of having been made by running the Program). +Whether that is true depends on what the Program does. + + 1. You may copy and distribute verbatim copies of the Program's +source code as you receive it, in any medium, provided that you +conspicuously and appropriately publish on each copy an appropriate +copyright notice and disclaimer of warranty; keep intact all the +notices that refer to this License and to the absence of any warranty; +and give any other recipients of the Program a copy of this License +along with the Program. + +You may charge a fee for the physical act of transferring a copy, and +you may at your option offer warranty protection in exchange for a fee. + + 2. You may modify your copy or copies of the Program or any portion +of it, thus forming a work based on the Program, and copy and +distribute such modifications or work under the terms of Section 1 +above, provided that you also meet all of these conditions: + + a) You must cause the modified files to carry prominent notices + stating that you changed the files and the date of any change. + + b) You must cause any work that you distribute or publish, that in + whole or in part contains or is derived from the Program or any + part thereof, to be licensed as a whole at no charge to all third + parties under the terms of this License. + + c) If the modified program normally reads commands interactively + when run, you must cause it, when started running for such + interactive use in the most ordinary way, to print or display an + announcement including an appropriate copyright notice and a + notice that there is no warranty (or else, saying that you provide + a warranty) and that users may redistribute the program under + these conditions, and telling the user how to view a copy of this + License. (Exception: if the Program itself is interactive but + does not normally print such an announcement, your work based on + the Program is not required to print an announcement.) + +These requirements apply to the modified work as a whole. If +identifiable sections of that work are not derived from the Program, +and can be reasonably considered independent and separate works in +themselves, then this License, and its terms, do not apply to those +sections when you distribute them as separate works. But when you +distribute the same sections as part of a whole which is a work based +on the Program, the distribution of the whole must be on the terms of +this License, whose permissions for other licensees extend to the +entire whole, and thus to each and every part regardless of who wrote it. + +Thus, it is not the intent of this section to claim rights or contest +your rights to work written entirely by you; rather, the intent is to +exercise the right to control the distribution of derivative or +collective works based on the Program. + +In addition, mere aggregation of another work not based on the Program +with the Program (or with a work based on the Program) on a volume of +a storage or distribution medium does not bring the other work under +the scope of this License. + + 3. You may copy and distribute the Program (or a work based on it, +under Section 2) in object code or executable form under the terms of +Sections 1 and 2 above provided that you also do one of the following: + + a) Accompany it with the complete corresponding machine-readable + source code, which must be distributed under the terms of Sections + 1 and 2 above on a medium customarily used for software interchange; or, + + b) Accompany it with a written offer, valid for at least three + years, to give any third party, for a charge no more than your + cost of physically performing source distribution, a complete + machine-readable copy of the corresponding source code, to be + distributed under the terms of Sections 1 and 2 above on a medium + customarily used for software interchange; or, + + c) Accompany it with the information you received as to the offer + to distribute corresponding source code. (This alternative is + allowed only for noncommercial distribution and only if you + received the program in object code or executable form with such + an offer, in accord with Subsection b above.) + +The source code for a work means the preferred form of the work for +making modifications to it. For an executable work, complete source +code means all the source code for all modules it contains, plus any +associated interface definition files, plus the scripts used to +control compilation and installation of the executable. However, as a +special exception, the source code distributed need not include +anything that is normally distributed (in either source or binary +form) with the major components (compiler, kernel, and so on) of the +operating system on which the executable runs, unless that component +itself accompanies the executable. + +If distribution of executable or object code is made by offering +access to copy from a designated place, then offering equivalent +access to copy the source code from the same place counts as +distribution of the source code, even though third parties are not +compelled to copy the source along with the object code. + + 4. You may not copy, modify, sublicense, or distribute the Program +except as expressly provided under this License. Any attempt +otherwise to copy, modify, sublicense or distribute the Program is +void, and will automatically terminate your rights under this License. +However, parties who have received copies, or rights, from you under +this License will not have their licenses terminated so long as such +parties remain in full compliance. + + 5. You are not required to accept this License, since you have not +signed it. However, nothing else grants you permission to modify or +distribute the Program or its derivative works. These actions are +prohibited by law if you do not accept this License. Therefore, by +modifying or distributing the Program (or any work based on the +Program), you indicate your acceptance of this License to do so, and +all its terms and conditions for copying, distributing or modifying +the Program or works based on it. + + 6. Each time you redistribute the Program (or any work based on the +Program), the recipient automatically receives a license from the +original licensor to copy, distribute or modify the Program subject to +these terms and conditions. You may not impose any further +restrictions on the recipients' exercise of the rights granted herein. +You are not responsible for enforcing compliance by third parties to +this License. + + 7. If, as a consequence of a court judgment or allegation of patent +infringement or for any other reason (not limited to patent issues), +conditions are imposed on you (whether by court order, agreement or +otherwise) that contradict the conditions of this License, they do not +excuse you from the conditions of this License. If you cannot +distribute so as to satisfy simultaneously your obligations under this +License and any other pertinent obligations, then as a consequence you +may not distribute the Program at all. For example, if a patent +license would not permit royalty-free redistribution of the Program by +all those who receive copies directly or indirectly through you, then +the only way you could satisfy both it and this License would be to +refrain entirely from distribution of the Program. + +If any portion of this section is held invalid or unenforceable under +any particular circumstance, the balance of the section is intended to +apply and the section as a whole is intended to apply in other +circumstances. + +It is not the purpose of this section to induce you to infringe any +patents or other property right claims or to contest validity of any +such claims; this section has the sole purpose of protecting the +integrity of the free software distribution system, which is +implemented by public license practices. Many people have made +generous contributions to the wide range of software distributed +through that system in reliance on consistent application of that +system; it is up to the author/donor to decide if he or she is willing +to distribute software through any other system and a licensee cannot +impose that choice. + +This section is intended to make thoroughly clear what is believed to +be a consequence of the rest of this License. + + 8. If the distribution and/or use of the Program is restricted in +certain countries either by patents or by copyrighted interfaces, the +original copyright holder who places the Program under this License +may add an explicit geographical distribution limitation excluding +those countries, so that distribution is permitted only in or among +countries not thus excluded. In such case, this License incorporates +the limitation as if written in the body of this License. + + 9. The Free Software Foundation may publish revised and/or new versions +of the General Public License from time to time. Such new versions will +be similar in spirit to the present version, but may differ in detail to +address new problems or concerns. + +Each version is given a distinguishing version number. If the Program +specifies a version number of this License which applies to it and "any +later version", you have the option of following the terms and conditions +either of that version or of any later version published by the Free +Software Foundation. If the Program does not specify a version number of +this License, you may choose any version ever published by the Free Software +Foundation. + + 10. If you wish to incorporate parts of the Program into other free +programs whose distribution conditions are different, write to the author +to ask for permission. For software which is copyrighted by the Free +Software Foundation, write to the Free Software Foundation; we sometimes +make exceptions for this. Our decision will be guided by the two goals +of preserving the free status of all derivatives of our free software and +of promoting the sharing and reuse of software generally. + + NO WARRANTY + + 11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY +FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN +OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES +PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED +OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS +TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE +PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, +REPAIR OR CORRECTION. + + 12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING +WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR +REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, +INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING +OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED +TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY +YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER +PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE +POSSIBILITY OF SUCH DAMAGES. + + END OF TERMS AND CONDITIONS + + How to Apply These Terms to Your New Programs + + If you develop a new program, and you want it to be of the greatest +possible use to the public, the best way to achieve this is to make it +free software which everyone can redistribute and change under these terms. + + To do so, attach the following notices to the program. It is safest +to attach them to the start of each source file to most effectively +convey the exclusion of warranty; and each file should have at least +the "copyright" line and a pointer to where the full notice is found. + + + Copyright (C) + + This program is free software; you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation; either version 2 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License along + with this program; if not, write to the Free Software Foundation, Inc., + 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. + +Also add information on how to contact you by electronic and paper mail. + +If the program is interactive, make it output a short notice like this +when it starts in an interactive mode: + + Gnomovision version 69, Copyright (C) year name of author + Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'. + This is free software, and you are welcome to redistribute it + under certain conditions; type `show c' for details. + +The hypothetical commands `show w' and `show c' should show the appropriate +parts of the General Public License. Of course, the commands you use may +be called something other than `show w' and `show c'; they could even be +mouse-clicks or menu items--whatever suits your program. + +You should also get your employer (if you work as a programmer) or your +school, if any, to sign a "copyright disclaimer" for the program, if +necessary. Here is a sample; alter the names: + + Yoyodyne, Inc., hereby disclaims all copyright interest in the program + `Gnomovision' (which makes passes at compilers) written by James Hacker. + + , 1 April 1989 + Ty Coon, President of Vice + +This General Public License does not permit incorporating your program into +proprietary programs. If your program is a subroutine library, you may +consider it more useful to permit linking proprietary applications with the +library. If this is what you want to do, use the GNU Lesser General +Public License instead of this License. diff --git a/README b/README new file mode 100644 index 0000000..3c9b6d2 --- /dev/null +++ b/README @@ -0,0 +1,91 @@ +PXE install w multi-boot, btrfs & Libreboot support + +Some things are specific to my home network, and uses files with secrets +that are not in this repo. Uses pxe or pxe-kexec (on libreboot, I have +not added a pxe rom, I use a minimal debian stable subvolume which acts +like a pxe rom). I use this for bare metal and vms, and two scripts +which can run post boot so I use them on vps distributed image as well. + +Features people may find useful: installs encrypted trisquel belenos, , +debian jessie, debian stretch, ubuntu 16.04, and arch (havne't done +recently, probably a bit broken), in a multi-boot setup using multiple +subvolumes of a single btrfs filesystem. Utilizes multiple disks, with +scripts to automatically decrypt on intentional reboots, but not after +shutdown or power loss. + +The partititioning and filesystem script is at +fai/config/hooks/partition.DEFAULT. Other debian based distros at least +as new as ubuntu 14.04 should work fine, and I'm planning to add Fedora +support. Disks are grouped as ssd or hdd and raided in raid 1 or raid 0 +per configuration. The base partitions are divided into boot, swap, and +root, (only boot is unencrypted). There are scripts to resize those +partitions post-provision and while the system is running. + +People who use fai may find these things as useful examples: it uses +dnsmasq (on a openwrt machine) for dhcp instead of the isc +dhcp. fai-wrapper is a small script to use basic fai classes outside of +fai. It does not use the fai partitioning tool, but the script is +inspired from it and works outside of fai. It supports running a fai +server on debian within android via Maru. + +It also automates configuration of an openwrt router after manual +initial installation. + +After provisionining is done, I sync files using btrfs, or unison for +vps, then automate further setup using a different set of scripts, +https://iankelling.org/git/?p=distro-setup;a=tree. + +My network is a wndr3700v2 router with openwrt on it and a few pcs/laptops. + +Since fai requires a debian server as the fai server, there are also +scripts to automate a debian install using pxe and preseeding, which can +be done from any distro. + +Some of the scripts have dependencies for some simple obvious utility +scripts from https://iankelling.org/git, and of course there are some +hostnames that are specific to my network. + +All scripts meant to be used directly are listed here: + + +# Scripts to setup the environment for the install + +arch-pxe # Setup arch pxe boot server from an arch base image +fai-redep # Deploy fai configuration to host "faiserver" +faiserver-revm # using pxe & preseed, create a vm which is a fai server +faiserver-uninstall # uninstall fai-server +faiserver-setup # install fai-server on the current machine +myfai-chboot # setup fai tftp and nfs. useful for doing pxe-kexec +pxe-server # disable/enable pxe dhcp, tfp, and nfs. calls myfai-chboot +wrt-setup-remote # setup my router in general: dhcp, dns, etc. + + +# Script to do a distro install + +dsfull # install & post-install a new fai distro +arch-init-remote # install arch after it's been booted into it's setup env +fai-kexec # Kexec this or a remote machine using host faiserver +live-kexec # fai kexec from upstream live cds, e.g. curl live-kexec|bash + + +# Test scripts + +arch-revm # test arch install on a fresh vm +fai-revm # test fai install on a fresh vm + + +# Scripts to call after a distro install for various reasons + +chboot # Set grub to boot into a different distro (installed earlier) +install-chboot # reinstall chboot to /boot subvols, for chboot updates. +eboot # reboot without automatic disk decryption +fai-wrapper # use fai classes outside of fai. sourced, not called. +faiserver-disable # Disable the fai nfs server exports +fresize # resize swap or boot partitions in a host + + +License stuff: +The license for the project is GPLv2 or later, mostly because fai is and +I periodically rebase off their example config, which contains small +scripts. Also, there is a modified encrypt.upstream, which is from the +cryptsetup package in arch, which is under the same license. diff --git a/arch-init b/arch-init new file mode 100755 index 0000000..2f14ad6 --- /dev/null +++ b/arch-init @@ -0,0 +1,114 @@ +#!/bin/bash -x +# Copyright (C) 2016 Ian Kelling + +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License +# as published by the Free Software Foundation; either version 2 +# of the License, or (at your option) any later version. + +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. + +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. + +x="$(readlink -f "$BASH_SOURCE")"; source "${x%/*}/bash-trace" +cd ${x%/*} + +export HOSTNAME="$1" +mirror=$2 + +(( $# >= 1 )) || { echo "$0: error: need 1 or 2 arguments"; exit 1; } + +rm -f /etc/pacman.d/mirrorlist +# https://wiki.archlinux.org/index.php/Mirrors#Sorting_mirrors + +if [[ $mirror ]]; then + echo "Server = $mirror" >> /etc/pacman.d/mirrorlist +fi +curl -s 'https://www.archlinux.org/mirrorlist/?country=US&protocol=https&ip_version=4&ip_version=6&use_mirror_status=on' | + sed -r 's/^[ #]*(Server *=)/\1/' >> /etc/pacman.d/mirrorlist + +. /a/bin/fai/fai-wrapper +export LUKS_DIR=/root/luks +export DISTRO=arch +partition_script=/a/bin/fai/fai/config/hooks/partition.DEFAULT +chmod +x $partition_script + +export PARTITION_PROMPT=true + +# to be idempotent if we fail after partitioning +already_partitioned=true +mount_out=$(mount) +for dir in /mnt{,/home,/boot,/a}; do + regex=" on $dir " + if [[ ! $mount_out =~ $regex ]]; then + already_partitioned=false + break + fi +done + +if ! $already_partitioned; then + /a/bin/fai/fai/config/hooks/partition.DEFAULT +fi + +. /tmp/fai/disk_var.sh + + +# arch doesn't need crypttab entries for initramfs crypt partititions +export rootn=1 +export bootn=3 +export swapn=2 +export BOOT_DEVICE +export ROOT_PARTITIONS +sed -ri --follow-symlinks "/^crypt_dev_\S+$rootn /d" /tmp/fai/crypttab + +if ! $already_partitioned; then + mount -o subvol=root_$DISTRO $ROOT_PARTITION /mnt + mkdir -p /mnt/boot + mount -o subvol=boot_$DISTRO $BOOT_PARTITION /mnt/boot +fi + +# https://wiki.archlinux.org/index.php/Dm-crypt/Device_encryption#Keyfiles +cp /root/luks/host-$HOSTNAME /mnt/crypto_keyfile.bin +chmod 600 /mnt/crypto_keyfile.bin + + +if [[ $mirror ]]; then + echo "$0: 404 errors about core.db etc are normal, +they will succeed using the secodary mirror" +fi +pacstrap /mnt base +cp /tmp/fai/{fstab,crypttab} /mnt/etc +cp /a/bin/fai/encrypt /mnt/usr/lib/initcpio/hooks + +cp -r /root/.ssh /mnt/root + +bindmount() { + local mountpoint=$2 + local source=$1 + mkdir -p $mountpoint + mount -o bind $source $mountpoint +} +bindmount /root/shadow /mnt/q/root/shadow +bindmount /a /mnt/a + +mkdir -p /mnt/etc/ssh +cp /etc/ssh/ssh_host_* /mnt/etc/ssh + +cp /a/bin/fai/arch-init-chroot /mnt/root +# for manual commands, arch-chroot /mnt bash +arch-chroot /mnt /root/arch-init-chroot + +# this gets mounted in chroot so we have to do it outside +rm -f /mnt/etc/resolv.conf +ln -s /run/systemd/resolve/resolv.conf /mnt/etc/resolv.conf + +# not necsesary, but makes reboot go fast. +umount -R /mnt; sleep 1 + +# causes 255 exit code, so doing this from the caller script. +# reboot now diff --git a/arch-init-chroot b/arch-init-chroot new file mode 100755 index 0000000..e173863 --- /dev/null +++ b/arch-init-chroot @@ -0,0 +1,169 @@ +#!/bin/bash -x +# Copyright (C) 2016 Ian Kelling + +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License +# as published by the Free Software Foundation; either version 2 +# of the License, or (at your option) any later version. + +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. + +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. +set -eE -o pipefail +trap 'echo "$0:$LINENO:error: \"$BASH_COMMAND\" returned $?" >&2' ERR + +# note, when I did genfstab, i +# neeeded to to do for x in mv /etc/*.pacorig; do mv $x ${x%.pacorig}; done + +pacman -Syu + +[[ -L /etc/localtime ]] || ln -s /usr/share/zoneinfo/America/Los_Angeles /etc/localtime +l=en_US.UTF-8 +echo "$l UTF-8" > /etc/locale.gen +locale-gen +echo "LANG=$l" > /etc/locale.conf +# if coming from windows, and you had to set the time, do this +# hwclock --systohc --utc + +# A password is required to access the volume: +# Command requires device and ampped name as arguments + +# If we were using btrfs raid, we supposedly would need this. +# # add btrfs as module instead of hook due to +# # https://wiki.archlinux.org/index.php/Btrfs, +# # https://bugs.archlinux.org/task/42884 +# # disabled, as with just the module, startup spammed something about +# # command takes a device name and something else. +# sed -ri --follow-symlinks '/^ *MODULES *=.*btrfs/!s/^( *MODULES *=.*)"/\1 btrfs"/' /etc/mkinitcpio.conf +# # remove extra space +# sed -ri --follow-symlinks 's/^( *MODULES *=[^"]*)" */\1"/' /etc/mkinitcpio.conf + + + + +# for desktop without full fs encryption, use this: +#cat > /etc/crypttab <<'EOF' +#tmp /dev/lvm/tmp /dev/urandom tmp,cipher=aes-xts-plain64,size=256 + +# otgherwise ERROR: file not found: `fsck.btrfs' +pacman -S --noconfirm btrfs-progs + +pacman -S --noconfirm grub gptfdisk + + +shopt -s extglob +echo "$0: fstab:" +cat /etc/fstab +# https://wiki.archlinux.org/index.php/Dm-crypt/System_configuration#Boot_loader +# if cryptdevice was lvm, it woulde be in this format, +# where x2-vg is from lvdisplay, VG Name field. +# cryptdevice=/dev/disk/by-uuid/585dff23-136f-446f-815f-01053b70c957:x2-vg +# but, if you are using your own fstab, it seems you just give it a name, +# which will be the crypt device name under /dev/mapper/ +# https://wiki.archlinux.org/index.php/GRUB#Additional_arguments + + +root_devs=( ${ROOT_PARTITIONS} ) +first_root_dev=${root_devs[0]} + + +k_args=( + cryptdevices=${ROOT_PARTITIONS// /,} + root=/dev/mapper/crypt_dev_${first_root_dev##*/} + resume=${first_root_dev%[0-9]}$swapn +) + + +# If we have more than 1 to decrypt, arch wiki lead me onto +# a sort of hacky way run the encrypt hook multiple times. + +# https://wiki.archlinux.org/index.php/Dm-crypt/Encrypting_an_entire_system#Configuring_mkinitcpio_2 +# used to have lvm2 after encrypt for lvm, but not using lvm anymore +for x in encrypt btrfs; do + sed -ri --follow-symlinks -f - /etc/mkinitcpio.conf < /etc/systemd/network/wired.network < /etc/systemd/network/br0.network < /etc/systemd/network/br0.netdev <&2' ERR + +usage() { + cat < /tmp/myarchinit.log +if ! ip a | grep '^ *inet ' | grep -vF 127.0.0.1; then + cat <<'eof' +We don't have an ipv4 address. Maybe arch doesn't do that for us, +or we are probably using an ethernet port +which is not the 1st one, so we haven't automatically done dhcpcd, +so let's do it on whatever interface has a carrier +eof + for f in /sys/class/net/*; do + if [[ `cat $f/carrier` == 1 ]]; then + echo $0: running: dhcpcd ${f##*/} + dhcpcd ${f##*/} + break + fi + done +fi +systemctl start sshd diff --git a/arch-pxe b/arch-pxe new file mode 100755 index 0000000..e480b5d --- /dev/null +++ b/arch-pxe @@ -0,0 +1,86 @@ +#!/bin/bash -lx +# Copyright (C) 2016 Ian Kelling + +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License +# as published by the Free Software Foundation; either version 2 +# of the License, or (at your option) any later version. + +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. + +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. + +# Setup arch pxe boot server from the base image. +# + +set -eE -o pipefail +trap 'echo "$0:$LINENO:error: \"$BASH_COMMAND\" returned $?" >&2' ERR + +usage() { + cat < airootfs.md5; popd + +# seems if you've done a pxe boot, mounted the nfs, +# then shutdown, it's still busy. +ssh wrt "/etc/init.d/nfsd stop; \ +{ ! mount | grep /run/archiso/bootmnt || umount /run/archiso/bootmnt; } && \ +rm -rf /mnt/usb/$iso" +scp -r $iso wrt:/mnt/usb +ssh wrt "cd /mnt/usb && rm -f tftpboot && ln -s $iso tftpboot" + +# The default settings in the installer expect to find the NFS at /run/archiso/bootmnt + +# background: great documentation at +# https://wiki.archlinux.org/index.php/PXE +# arch can do netboot like ubuntu etc, but the docs look a little +# complicated, so fuck it, we use nfs cuz it's easy + +rm -rf $iso +s rm -rf squashfs-root diff --git a/arch-revm b/arch-revm new file mode 120000 index 0000000..de8dd4b --- /dev/null +++ b/arch-revm @@ -0,0 +1 @@ +fai-revm \ No newline at end of file diff --git a/bash-trace b/bash-trace new file mode 100644 index 0000000..61f8ae5 --- /dev/null +++ b/bash-trace @@ -0,0 +1,48 @@ +# meant to be sourced. copy/pasted from https://iankelling.org/git/?p=errhandle;a=summary + +bash-trace() { + local -i argc_index=0 arg frame i start=${1:-1} max_indent=8 indent + local source + local extdebug=false + if [[ $(shopt -p extdebug) == *-s* ]]; then + extdebug=true + fi + + for ((frame=0; frame < ${#FUNCNAME[@]}-1; frame++)); do + argc=${BASH_ARGC[frame]} + argc_index+=$argc + ((frame < start)) && continue + if (( ${#BASH_SOURCE[@]} > 1 )); then + source="${BASH_SOURCE[frame+1]}:${BASH_LINENO[frame]}:" + fi + indent=$((frame-start+1)) + indent=$((indent < max_indent ? indent : max_indent)) + printf "%${indent}s↳%sin \`%s" '' "$source" "${FUNCNAME[frame]}" + if $extdebug; then + for ((i=argc_index-1; i >= argc_index-argc; i--)); do + printf " %s" "${BASH_ARGV[i]}" + done + fi + echo \' + done +} + + +errcatch() { + set -E; shopt -s extdebug + _err-trap() { + err=$? + exec >&2 + set +x + echo "${BASH_SOURCE[1]}:${BASH_LINENO[0]}:in \`$BASH_COMMAND' returned $err" + bash-trace 2 + set -e + "${_errcatch_cleanup[@]}" + echo "$0: exiting with code $err" + exit $err + } + trap _err-trap ERR + set -o pipefail +} + +errcatch diff --git a/chboot b/chboot new file mode 120000 index 0000000..b311a15 --- /dev/null +++ b/chboot @@ -0,0 +1 @@ +fai/config/files/boot/chboot/DEFAULT \ No newline at end of file diff --git a/chost b/chost new file mode 100755 index 0000000..6168741 --- /dev/null +++ b/chost @@ -0,0 +1,13 @@ +#!/bin/bash +# Copyright (C) 2016 Ian Kelling + +# chost: get canonical hostname + +set -eE -o pipefail +trap 'echo "$0:$LINENO:error: \"$BASH_COMMAND\" returned $?" >&2' ERR + +host=$1 +addr=$(host $host | sed -rn 's/^\S+ has address //p;T;q') +h=$(host $addr) +h=${h##* } +echo ${h%%.*} diff --git a/debian-preseed b/debian-preseed new file mode 100755 index 0000000..ea1e847 --- /dev/null +++ b/debian-preseed @@ -0,0 +1,144 @@ +#!/bin/bash +# Copyright (C) 2016 Ian Kelling + +set -eE -o pipefail +trap 'echo "$0:$LINENO:error: \"$BASH_COMMAND\" returned $?" >&2' ERR + +usage() { + cat <>$preseed + ;; + debian-jessie) + wget -q https://www.debian.org/releases/jessie/$preseed + wget -qN http://ftp.nl.debian.org/debian/dists/jessie/$neboot_path + cat >>$preseed <<'EOF' +tasksel tasksel/first multiselect ssh-server +EOF + if ! $interactive_partition; then + cat >>$preseed <> $preseed <> $preseed <> /home/$user/.ssh/authorized_keys"; \ +in-target chown -R $user:$user /home/$user; \ +in-target chmod -R go-rwx /home/$user/.ssh/authorized_keys; \ +in-target cp -r /home/$user/.ssh /root; \ +in-target usermod -a -G sudo $user; +EOF diff --git a/debian-pxe-preseed b/debian-pxe-preseed new file mode 100755 index 0000000..31a038a --- /dev/null +++ b/debian-pxe-preseed @@ -0,0 +1,45 @@ +#!/bin/bash +# Copyright (C) 2016 Ian Kelling + +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License +# as published by the Free Software Foundation; either version 2 +# of the License, or (at your option) any later version. + +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. + +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. + + +x="$(readlink -f "$BASH_SOURCE")"; source "${x%/*}/bash-trace" + +[[ $EUID == 0 ]] || exec sudo "$BASH_SOURCE" "$@" + +src=$(readlink -f "${BASH_SOURCE%/*}") + +e() { echo "$*"; "$@"; } + +mount_dir=$(mktemp -d) + +cleanup() { cd; umount -f $mount_dir; } +_errcatch_cleanup=cleanup +e mount -o users wrt:/mnt/usb $mount_dir + + +cd $mount_dir +e rm -rf debian-wheezy +mkdir debian-wheezy +cd debian-wheezy +e $src/debian-preseed "$@" # my script +cd .. +e rm -f tftpboot +e ln -s debian-wheezy tftpboot + +cd / +e umount $mount_dir +e $src/pxe-server default plain # my script diff --git a/devbyid b/devbyid new file mode 120000 index 0000000..9a02442 --- /dev/null +++ b/devbyid @@ -0,0 +1 @@ +fai/config/distro-install-common/devbyid \ No newline at end of file diff --git a/dsfull b/dsfull new file mode 100755 index 0000000..29946f3 --- /dev/null +++ b/dsfull @@ -0,0 +1,104 @@ +#!/bin/bash -l +# Copyright (C) 2016 Ian Kelling + +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at + +# http://www.apache.org/licenses/LICENSE-2.0 + +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +x="$(readlink -f "$BASH_SOURCE")"; source "${x%/*}/bash-trace" + +reboot=true +if [[ $1 == -r ]]; then + reboot=false + shift +fi + +usage() { + cat </dev/null 2>&1 + [ "${quiet}" = "y" ] && CSQUIET=">/dev/null" + + # Get keyfile if specified + ckeyfile="/crypto_keyfile.bin" + if [ -n "$cryptkey" ]; then + IFS=: read ckdev ckarg1 ckarg2 </dev/null 2>&1 + umount /ckey + ;; + *) + # Read raw data from the block device + # ckarg1 is numeric: ckarg1=offset, ckarg2=length + dd if="$resolved" of="$ckeyfile" bs=1 skip="$ckarg1" count="$ckarg2" >/dev/null 2>&1 + ;; + esac + fi + [ ! -f ${ckeyfile} ] && echo "Keyfile could not be opened. Reverting to passphrase." + fi + + for cryptdev in ${cryptdevices//,/ }; do + cryptname=crypt_dev_${cryptdev##*/} + + if [ -n "${cryptoptions}" ]; then + cryptargs="${cryptargs} --allow-discards" + fi + for cryptopt in ${cryptoptions//,/ }; do + case ${cryptopt} in + no-allow-discards) + cryptargs="" + ;; + *) + echo "Encryption option '${cryptopt}' not known, ignoring." >&2 + ;; + esac + done + + if resolved=$(resolve_device "${cryptdev}" ${rootdelay}); then + if cryptsetup isLuks ${resolved} >/dev/null 2>&1; then + dopassphrase=1 + # If keyfile exists, try to use that + if [ -f ${ckeyfile} ]; then + if eval cryptsetup --key-file ${ckeyfile} open --type luks ${resolved} ${cryptname} ${cryptargs} ${CSQUIET}; then + dopassphrase=0 + else + echo "Invalid keyfile. Reverting to passphrase." + fi + fi + # Ask for a passphrase + if [ ${dopassphrase} -gt 0 ]; then + echo "" + echo "A password is required to access the ${cryptname} volume:" + + #loop until we get a real password + while ! eval cryptsetup open --type luks ${resolved} ${cryptname} ${cryptargs} ${CSQUIET}; do + sleep 2; + done + fi + if [ -e "/dev/mapper/${cryptname}" ]; then + if [ ${DEPRECATED_CRYPT} -eq 1 ]; then + export root="/dev/mapper/root" + fi + else + err "Password succeeded, but ${cryptname} creation failed, aborting..." + exit 1 + fi + elif [ -n "${crypto}" ]; then + msg "Non-LUKS encrypted device found..." + if echo "$crypto" | awk -F: '{ exit(NF == 5) }'; then + err "Verify parameter format: crypto=hash:cipher:keysize:offset:skip" + err "Non-LUKS decryption not attempted..." + return 1 + fi + exe="cryptsetup open --type plain $resolved $cryptname $cryptargs" + IFS=: read c_hash c_cipher c_keysize c_offset c_skip </dev/null 2>&1 + [ "${quiet}" = "y" ] && CSQUIET=">/dev/null" + + # Get keyfile if specified + ckeyfile="/crypto_keyfile.bin" + if [ -n "$cryptkey" ]; then + IFS=: read ckdev ckarg1 ckarg2 </dev/null 2>&1 + umount /ckey + ;; + *) + # Read raw data from the block device + # ckarg1 is numeric: ckarg1=offset, ckarg2=length + dd if="$resolved" of="$ckeyfile" bs=1 skip="$ckarg1" count="$ckarg2" >/dev/null 2>&1 + ;; + esac + fi + [ ! -f ${ckeyfile} ] && echo "Keyfile could not be opened. Reverting to passphrase." + fi + + if [ -n "${cryptdevice}" ]; then + DEPRECATED_CRYPT=0 + IFS=: read cryptdev cryptname cryptoptions <&2 + ;; + esac + done + + if resolved=$(resolve_device "${cryptdev}" ${rootdelay}); then + if cryptsetup isLuks ${resolved} >/dev/null 2>&1; then + [ ${DEPRECATED_CRYPT} -eq 1 ] && warn_deprecated + dopassphrase=1 + # If keyfile exists, try to use that + if [ -f ${ckeyfile} ]; then + if eval cryptsetup --key-file ${ckeyfile} open --type luks ${resolved} ${cryptname} ${cryptargs} ${CSQUIET}; then + dopassphrase=0 + else + echo "Invalid keyfile. Reverting to passphrase." + fi + fi + # Ask for a passphrase + if [ ${dopassphrase} -gt 0 ]; then + echo "" + echo "A password is required to access the ${cryptname} volume:" + + #loop until we get a real password + while ! eval cryptsetup open --type luks ${resolved} ${cryptname} ${cryptargs} ${CSQUIET}; do + sleep 2; + done + fi + if [ -e "/dev/mapper/${cryptname}" ]; then + if [ ${DEPRECATED_CRYPT} -eq 1 ]; then + export root="/dev/mapper/root" + fi + else + err "Password succeeded, but ${cryptname} creation failed, aborting..." + exit 1 + fi + elif [ -n "${crypto}" ]; then + [ ${DEPRECATED_CRYPT} -eq 1 ] && warn_deprecated + msg "Non-LUKS encrypted device found..." + if echo "$crypto" | awk -F: '{ exit(NF == 5) }'; then + err "Verify parameter format: crypto=hash:cipher:keysize:offset:skip" + err "Non-LUKS decryption not attempted..." + return 1 + fi + exe="cryptsetup open --type plain $resolved $cryptname $cryptargs" + IFS=: read c_hash c_cipher c_keysize c_offset c_skip <&2' ERR + +[[ $EUID == 0 ]] || exec sudo "$BASH_SOURCE" "$@" + +usage() { + cat <<'EOF' +usage: $0 [-h|--help] [SERVER] +Kexec this or a remote machine using host faiserver + +If SERVER argument, ssh to root@SERVER before doing kexec. This does +what pxe would do, but skipping boot sequence up to and including the +pxe dhcp. + +EOF + exit $1 +} +case $1 in + -h|--help) usage ;; +esac + + +if [[ $1 ]]; then + prefix="ssh root@$1" +fi +$prefix touch /tmp/keyscript-off +$prefix pxe-kexec -n --ignore-whitelist -l fai-generated faiserver diff --git a/fai-redep b/fai-redep new file mode 100755 index 0000000..fabada0 --- /dev/null +++ b/fai-redep @@ -0,0 +1,134 @@ +#!/bin/bash -l +# Copyright (C) 2016 Ian Kelling + +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License +# as published by the Free Software Foundation; either version 2 +# of the License, or (at your option) any later version. + +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. + +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. +set -eE -o pipefail +trap 'echo "$0:$LINENO:error: \"$BASH_COMMAND\" returned $?" >&2' ERR + +x="$(readlink -f "$BASH_SOURCE")"; cd ${x%/*} + +usage() { + cat <>$f +fi + +if ! modprobe nfsd &>/dev/null; then + # no apt-cache on maru debian, because we are low on space already + sed -i '/^ *APTPROXY=/d' /srv/fai/config/class/DEBIAN.var + # maru debian doesn't have loopback devs created + if ! losetup -f; then + shopt -s nullglob + x=(/dev/loop*) + minor=0 + if (( ${#x[@]} )); then + minor=$(( ${x[-1]#/dev/loop} + 1 )) + fi + mknod -m660 /dev/loop$minor b 7 $minor + losetup -f + fi + # -B boo only iso, no nfsroot, no paritial miorr, no config space. + # -f = force, for overwriting + # -S = make squash image for http booting + # -d config space url, instead of putting it in the squash.img, + # this just makes it so that we don't have to regenerate the img + # when the config changes. + cd /srv/fai/config + tar czf /var/www/faiserver/html/config.tar.gz . + if $changed || [[ ! -e /var/www/faiserver/html/squash.img ]]; then + # note, on maru, selinux needs to be disabled in android before + # this will work. + mount + export debug=true + fai-cd -d http://faiserver:8080/config.tar.gz -f -M -S /var/www/faiserver/html/squash.img + mount + fi +fi +EOF diff --git a/fai-revm b/fai-revm new file mode 100755 index 0000000..3b05795 --- /dev/null +++ b/fai-revm @@ -0,0 +1,141 @@ +#!/bin/bash -l +# Copyright (C) 2016 Ian Kelling + +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License +# as published by the Free Software Foundation; either version 2 +# of the License, or (at your option) any later version. + +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. + +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. + + + +x="$(readlink -f "$BASH_SOURCE")"; source "${x%/*}/bash-trace" + +script_dir="$(readlink -f "$BASH_SOURCE")" + +e() { echo "$*"; "$@"; } + + +usage() { + cat < 1 should be the only +# important things to test. +disk_count=1 + + +if [[ $script_dir == /a/bin/* ]]; then + # Copy our script elsewhere so we can develop it + # and save it at the same time it's running + rm -rf /tmp/faifreeze + mkdir -p /a/tmp + cp -ar /a/bin/fai /tmp/faifreeze + exec /tmp/faifreeze/${BASH_SOURCE##*/} "${orig_args[@]}" +fi + +cd $script_dir + +is_arch_revm() { + [[ ${0##*/} == arch-revm ]] +} + +cleanup() { + ./pxe-server + ./faiserver-disable +} +_errcatch_cleanup=cleanup + +if is_arch_revm; then + ./pxe-server demohost arch + sleep 2 + # via osinfo-query os. guessing arch is closest to latest fedora. + variant=fedora22 +else + ./pxe-server demohost fai + sleep 2 + # I don't think these variants actually make a diff for us, but I + # use the appropriate one when trying a new distro just in case. + variant=ubuntu14.04 + #variant=ubuntu16.04 + #variant=debian8 +fi + +name=demohost + +e s virshrm $name ||: + + +disk_arg=() +for ((i=1; i <= disk_count; i++)); do + f=/var/lib/libvirt/images/${name}$i + disk_arg+=("--disk path=$f") + if $new_disk || [[ ! -e $f ]]; then + s rm -f $f + e s qemu-img create -o preallocation=metadata -f qcow2 $f 50G + fi +done + +if [[ $SSH_CLIENT ]]; then + console_arg=--noautoconsole +fi + +# --cpu host: this causes mkfs.btrfs to fail with a stack trace which began +# something like: +# init_module+0x108/0x1000 [raid6_pq] +# +# uniq is to stop gtk-warning spam +e s virt-install --os-variant $variant -n $name --pxe -r 2048 --vcpus 1 \ + ${disk_arg[*]} -w bridge=br0,mac=52:54:00:9c:ef:ad $reboot_arg \ + --graphics spice,listen=0.0.0.0 $console_arg |& grep -v '^ *$' | uniq & + +if [[ $SSH_CLIENT ]]; then + fg +fi + +sleep 30 +while ! timeout -s 9 10 ssh -oBatchMode=yes root@$name /bin/true; do + e sleep 5 +done +unset _errcatch_cleanup +e pxe-server +if is_arch_revm; then + ./arch-init-remote $name +fi diff --git a/fai-wrapper b/fai-wrapper new file mode 100644 index 0000000..04625fd --- /dev/null +++ b/fai-wrapper @@ -0,0 +1,44 @@ +#!/bin/bash +# Copyright (C) 2016 Ian Kelling + +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License +# as published by the Free Software Foundation; either version 2 +# of the License, or (at your option) any later version. + +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. + +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. + +# For using some fai commands outside of fai. +# Usually this is sourced from another script. Note this has +# paths specific to Ian's machine. +# to set fai classes, export CLASS_CLASSNAME=true +ifclass() { + local var=${1/#/CLASS_} + [[ $HOSTNAME == $1 || ${!var} ]] +} +fai-setclass() { + for class in "$@"; do + # export class vars with CLASS_ in front to avoid name collisions. + classes+=" $class" + export CLASS_$class=true + done + classes="${classes# }" + export classes +} +eval-fai-classfile() { + file=$1 + fai-setclass $(bash -l $file) +} +export -f ifclass +classes=DEFAULT # used by fcopy +export CLASS_DEFAULT=true +eval-fai-classfile /a/bin/fai/fai/config/class/50-host-classes +export FAI_ROOT=/ +export FAI=/a/bin/fai/fai/config diff --git a/fai/config/basefiles/mk-basefile b/fai/config/basefiles/mk-basefile index ebbe3d9..d9823cf 100755 --- a/fai/config/basefiles/mk-basefile +++ b/fai/config/basefiles/mk-basefile @@ -17,16 +17,18 @@ # Define your local mirros here # For the first stage, set the CentOS/SLC mirror in /etc/rinse/rinse.conf -MIRROR_DEBIAN=http://httpredir.debian.org/debian/ +MIRROR_DEBIAN=http://http.us.debian.org/debian MIRROR_UBUNTU=http://mirror.netcologne.de/ubuntu/ +MIRROR_TRISQUEL=http://mirror.fsf.org/trisquel/ MIRROR_CENTOS=http://mirror.netcologne.de/ EXCLUDE_SQUEEZE=isc-dhcp-client,isc-dhcp-common,info,tasksel,tasksel-data EXCLUDE_WHEEZY=info,tasksel,tasksel-data EXCLUDE_JESSIE=tasksel,tasksel-data EXCLUDE_STRETCH=tasksel,tasksel-data - +EXCLUDE_BELENOS=dhcp3-client,dhcp3-common,info EXCLUDE_TRUSTY=dhcp3-client,dhcp3-common,info +EXCLUDE_FLIDAS=tasksel,tasksel-data EXCLUDE_XENIAL=tasksel,tasksel-data INCLUDE_DEBIAN=aptitude @@ -261,6 +263,8 @@ case "$target" in SLC6_32) slc i386 6 ;; SLC6_64) slc amd64 6 ;; SLC7_64) slc amd64 7 ;; + BELENOS*|FLIDAS*) + debgeneric $target $MIRROR_TRISQUEL ;; TRUSTY*|XENIAL*) debgeneric $target $MIRROR_UBUNTU ;; SQUEEZE*|WHEEZY*|JESSIE*|STRETCH*) diff --git a/fai/config/class/50-host-classes b/fai/config/class/50-host-classes index fc89c1d..465a64f 100755 --- a/fai/config/class/50-host-classes +++ b/fai/config/class/50-host-classes @@ -1,31 +1,95 @@ -#! /bin/bash +#!/bin/bash -l # assign classes to hosts based on their hostname +# NOTE: +# 51-multi-boot should have something like this +# for transient host configs which are not saved in +# git (and make it executable): + +# if [[ ! -e /a/bin/fai/fai-wrapper ]]; then +# case $HOSTNAME in +# frodo) echo STABLE ;; +# esac +# fi + + # do not use this if a menu will be presented [ "$flag_menu" ] && exit 0 -# use a list of classes for our demo machine -case $HOSTNAME in - faiserver) - echo "FAIBASE DEBIAN DEMO FAISERVER" ;; - demohost|client*) - echo "FAIBASE DEBIAN DEMO" ;; - xfcehost) - echo "FAIBASE DEBIAN DEMO XORG XFCE LVM";; - gnomehost) - echo "FAIBASE DEBIAN DEMO XORG GNOME";; - centos) - echo "FAIBASE CENTOS" # you may want to add class XORG here - ifclass I386 && echo CENTOS6_32 # AFAIK there's no 32bit C7 - ifclass AMD64 && echo CENTOS7_64 - exit 0 ;; # CentOS does not use the GRUB class - slchost) - # Scientific Linux Cern, is very similar to CentOS. SLC should alsways use the class CENTOS - echo "FAIBASE CENTOS SLC" # you may want to add class XORG here - ifclass I386 && echo SLC7_32 - ifclass AMD64 && echo SLC7_64 - exit 0 ;; # CentOS/SLC does not use the GRUB class - *) - echo "FAIBASE DEBIAN DEMO" ;; -esac + +# For multi-boot system. +# We check that we aren't in a pxe boot environment. +# There is probably a better way to do this. +# We check the reverse condition in 51-multi-boot, +# and set what os we are installing, but don't check it +# into git since it changes regularly. +# It's code looks like this: +# if [[ ! -e /a/bin/fai/fai-wrapper ]]; then +# case $HOSTNAME in +# tp) DEBIAN STABLE VOL_STABLE STABLE_FREE;; +# # add more multi-boot hostnames here +# esac +# fi +# +# Each host defines the base distro: UBUNTU or DEBIAN. +# the disto version, also the basefile name if we aren't installing debian stable: +# STABLE, STRETCH64, XENIAL64, BELENOS64, FLIDAS64 +# the distro subvol name, we can add as many of these as we want: +# VOL_STABLE, VOL_STABLE_BOOTSTRAP, VOL_STRETCH, VOL_XENIAL, VOL_BELENOS, VOL_FLIDAS +# Using VOL_STABLE_BOOTSTRAP sets up the install to act like a pxe rom if +# grub sets a specific var. +# and the class which defines the apt sources files we want, +# STABLE_FREE, STABLE_NONFREE, TESTING_FREE, TESTING_NONFREE, +# XENIAL_FREE (no XENIAL_NONFREE setup yet), BELENOS, FLIDAS, STABLE_LINODE. +# This is a little redundant in some cases, but it keeps things +# simpler. +# +# +# Other notable classes: +# +# REPARTITION: we try to reuse partitions/filesystems to install a new +# os into a multi-os system, if we see some basic hueristics, like the +# right amount of them. This overrides that. +# +# PARTITION_PROMPT: If we don't see partitions to reuuse, prompt +# to make sure we really want to repartition and use a completely +# fresh install. I use this in case our repartition check has +# a bug in it, or I accidentally set REPARTITION. +# +# ROTATIONAL: in a system with ssd and hdd, install to the hdd +# instead of the default ssd. +# +# RAID0: Use raid 0 even if there are >= 4 disks with boot partititions. +# +# +if [[ -e /a/bin/fai/fai-wrapper ]]; then + source /a/bin/distro-functions/src/identify-distros + if isdebian; then + echo "DEBIAN" + fi + if isdebian-stable; then + echo "STABLE" + case $HOSTNAME in + li|lj) echo "STABLE_LINODE" ;; + *) + # nonfree repo is not going away any time soon due to + # gcc-doc being in nonfree + echo "STABLE_NONFREE" + ;; + esac + elif isdebian-testing; then + echo "TESTING_NONFREE" + fi +fi + +echo "FAIBASE" + +echo "PARTITION_PROMPT" +#echo REPARTITION + + +if grep ^52:54:00: /sys/class/net/eth0/address &>/dev/null; then + # if our eth0 mac is in the kvm range, we are a vm. + echo "VM" +fi diff --git a/fai/config/class/DEBIAN.var b/fai/config/class/DEBIAN.var index a6fb66b..ae445fa 100644 --- a/fai/config/class/DEBIAN.var +++ b/fai/config/class/DEBIAN.var @@ -1,3 +1,4 @@ +#### from upstream example config, except where noted CONSOLEFONT= KEYMAP=us-latin1 @@ -9,7 +10,9 @@ MODULESLIST="usbhid psmouse" # if you have enough RAM (>2GB) you may want to enable this line. It # also puts /var/cache into a ramdisk. -#FAI_RAMDISKS="$target/var/lib/dpkg $target/var/cache" +# ian: uncommented +FAI_RAMDISKS="$target/var/lib/dpkg $target/var/cache" # if you want to use the faiserver as APT proxy -#APTPROXY=http://faiserver:3142 +# ian: uncommented +APTPROXY=http://faiserver:3142 diff --git a/fai/config/class/DEFAULT.var b/fai/config/class/DEFAULT.var new file mode 100644 index 0000000..9934bb4 --- /dev/null +++ b/fai/config/class/DEFAULT.var @@ -0,0 +1,9 @@ +# according to fai-guide, required to enable saving logs +# remotely. + +LOGUSER=fai + +# when downloading from https intead of nfs, this is not set, +# it is used as the default for LOGSERVER, and for calling chboot. +# My faiserver's hostname is always faiserver, so just hardcoding it. +SERVER=faiserver \ No newline at end of file diff --git a/fai/config/class/FAIBASE.var b/fai/config/class/FAIBASE.var index 789cc7f..f02f58a 100644 --- a/fai/config/class/FAIBASE.var +++ b/fai/config/class/FAIBASE.var @@ -1,3 +1,5 @@ +#### from upstream example config, except where noted + # default values for installation. You can override them in your *.var files # allow installation of packages from unsigned repositories @@ -5,11 +7,8 @@ FAI_ALLOW_UNSIGNED=1 # Set UTC=yes if your system clock is set to UTC (GMT), and UTC=no if not. UTC=yes -TIMEZONE=Europe/Berlin - -# the hash of the root password for the new installed linux system -# pw is "fai" -ROOTPW='$1$kBnWcO.E$djxB128U7dMkrltJHPf6d1' +## changed from upstream. found in /usr/share/zoneinfo/, via fai-guide +TIMEZONE=US/Pacific # errors in tasks greater than this value will cause the installation to stop STOP_ON_ERROR=700 diff --git a/fai/config/class/UBUNTU.var b/fai/config/class/UBUNTU.var deleted file mode 100644 index f45c3ce..0000000 --- a/fai/config/class/UBUNTU.var +++ /dev/null @@ -1 +0,0 @@ -ubuntudist=xenial diff --git a/fai/config/class/UBUNTU.var b/fai/config/class/UBUNTU.var new file mode 120000 index 0000000..702cb15 --- /dev/null +++ b/fai/config/class/UBUNTU.var @@ -0,0 +1 @@ +DEBIAN.var \ No newline at end of file diff --git a/fai/config/class/example.profile b/fai/config/class/example.profile deleted file mode 100644 index 75ff37f..0000000 --- a/fai/config/class/example.profile +++ /dev/null @@ -1,54 +0,0 @@ -Default: Xfce - -Name: Simple -Description: My first FAI installation -Short: just a very simple example, no xorg, an account called demo -Long: This is the demohost example of FAI. -Additional account called demo with password: fai, root password: fai -All needed packages are already on the CD or USB stick. -Classes: INSTALL FAIBASE DEBIAN DEMO - -Name: Xfce -Description: Xfce desktop, LVM partitioning -Short: A fancy Xfce desktop will be installed, the user account is demo -Long: This is the Xfce desktop example. Additional account called -demo with password: fai, root password: fai -All needed packages are already on the CD or USB stick. -Classes: INSTALL FAIBASE DEBIAN DEMO XORG XFCE LVM - -Name: Gnome -Description: Gnome desktop installation -Short: A Gnome desktop, no LVM, You will get an account called demo -Long: This is the Gnome desktop example. Additional account called -demo with password: fai, root password: fai -You should have a fast network connection, because most packages are -downloaded from the internet. -Classes: INSTALL FAIBASE DEBIAN DEMO XORG GNOME - -Name: CentOS 7 -Description: CentOS 7 with Xfce desktop -Short: A normal Xfce desktop, running CentOS 7 -Long: We use the Debian nfsroot for installing the CentOS 7 OS. -You should have a fast network connection, because most packages are -downloaded from the internet. -Classes: INSTALL FAIBASE CENTOS CENTOS7_64 XORG - -Name: Ubuntu -Description: Ubuntu 16.04 desktop installation -Short: Unity desktop -Long: We use the Debian nfsroot for installing the Ubuntu OS. -You should have a fast network connection, because most packages are -downloaded from the internet. -Classes: INSTALL FAIBASE DEMO DEBIAN UBUNTU XENIAL XENIAL64 XORG - -Name: Inventory -Description: Show hardware info -Short: Show some basic hardware infos -Long: Execute commands for showing hardware info -Classes: INVENTORY - -Name: Sysinfo -Description: Show defailed system information -Short: Show detailed hardware and system information -Long: Execute a lot of commands for collecting system information -Classes: SYSINFO diff --git a/fai/config/disk_config/VM b/fai/config/disk_config/VM new file mode 100644 index 0000000..53c6527 --- /dev/null +++ b/fai/config/disk_config/VM @@ -0,0 +1,2 @@ +disk_config disk1 disklabel:gpt-bios bootable:1 fstabkey:uuid +primary / 100% ext4 noatime,errors=remount-ro diff --git a/fai/config/distro-install-common/devbyid b/fai/config/distro-install-common/devbyid new file mode 100755 index 0000000..e344389 --- /dev/null +++ b/fai/config/distro-install-common/devbyid @@ -0,0 +1,19 @@ +#!/bin/bash + +# input eg: /dev/sda1 or /dev/sda +# output: /dev/disk/by-id/model+serial, or if no link exists, the same as input + +short_dev=$1 + +# devices are identified by model+serial num, +# and wwn. model+serial gives me more info, so use that. +shopt -s extglob +for id in /dev/disk/by-id/!(wwn*); do + [[ -e $id ]] || break # if we matched nothing + if [[ $(readlink -f $id) == "$short_dev" ]]; then + printf '%s\n' "$id" + exit + fi +done +# a vm may not have a by-id link. +printf '%s\n' "$short_dev" diff --git a/fai/config/distro-install-common/end b/fai/config/distro-install-common/end new file mode 100755 index 0000000..cb953a4 --- /dev/null +++ b/fai/config/distro-install-common/end @@ -0,0 +1,99 @@ +#!/bin/bash -x + +set -eE -o pipefail +trap 'echo "$0:$LINENO:error: \"$BASH_COMMAND\" returned $?" >&2' ERR + +if [[ $EUID != 0 ]]; then + echo "$0: error: expected to be root." + exit 1 +fi + +# ssh host keys +# note, $BASH_SOURCE is not defined here under fai. +src=$(dirname "$0")/p/c/machine_specific/$HOSTNAME/filesystem/etc/ssh +dst=$target/etc/ssh +if [[ -e $src && -e $dst ]]; then + # outside of fai context, we skip this + cp -rT $src $dst +fi + +if ifclass VOL_STABLE_BOOTSTRAP; then + exit 0 +fi + +TPW=/q/root/shadow/traci-simple +if ifclass tp; then + ROOTPW="$TPW" +else + ROOTPW=/q/root/shadow/standard +fi + +chpw() { + # generating a hashed password: + # under debian, you can do + # mkpasswd -m sha-512 -s >/q/root/shadow/standard + # On arch, best seems to be copy your shadow file to a temp location, + # then passwd, get out the new pass, then copy the shadow file back. + + user=$1 + pwfile=$2 + if [[ $pwfile && -e $pwfile ]]; then + printf "$user:" | cat - "$pwfile" | $ROOTCMD chpasswd -e + else + echo "$0: warning: no pw set for $user" >&2 + fi +} +au() { # add user + if ! $ROOTCMD getent passwd ${@: -1}; then + $ROOTCMD useradd -m -s /bin/bash $@ + fi +} + +chpw root "$ROOTPW" +# 9 = user already exists. so we are idempotent. +au iank +chpw iank "$ROOTPW" + +au traci +if ifclass frodo; then + chpw traci "$TPW" +fi +# comparing iank's groups to traci, I see none she should join on arch +$ROOTCMD usermod -a -G traci iank + +$ROOTCMD getent group docker &>/dev/null || $ROOTCMD groupadd -r docker +$ROOTCMD usermod -a -G docker iank + +# based on unison error, with 8192 from +# sysctl -a | grep fs.inotify.max_user_watches +#http://stackoverflow.com/questions/535768/what-is-a-reasonable-amount-of-inotify-watches-with-linux + +f=$target/etc/sysctl.d/99-sysctl.conf +key=fs.inotify.max_user_watches +if [[ -e $f ]]; then sed -ri --follow-symlinks "/^\s*$key\s*=/d" $f; fi +echo "fs.inotify.max_user_watches = 1000000" >> $f +# applies it. it would be also be applied after a reboot +$ROOTCMD sysctl --system + +f=$target/etc/sudoers +line='iank ALL=(ALL) NOPASSWD: ALL' +if [[ ! -e $f ]] || ! grep -xF "$line" $f; then + echo "$line" >> $f +fi + +dir=/p/c/machine_specific/$HOSTNAME/.unison +$ROOTCMD mkdir -p $dir +if ! $ROOTCMD test -L /root/.unison; then + $ROOTCMD rm -rf /root/.unison + $ROOTCMD ln -s -T $dir /root/.unison +fi + +$ROOTCMD chown -R 1000:1000 $dir +while true; do + $ROOTCMD chown 1000:1000 $dir + $ROOTCMD chmod 700 $dir + dir=$(dirname $dir) + if [[ $dir == /p ]]; then break; fi +done + +au -s /bin/false --home-dir /var/lib/bitcoind bitcoin diff --git a/fai/config/distro-install-common/libreboot_grub.cfg b/fai/config/distro-install-common/libreboot_grub.cfg new file mode 100644 index 0000000..69e1c52 --- /dev/null +++ b/fai/config/distro-install-common/libreboot_grub.cfg @@ -0,0 +1,70 @@ +#!/bin/sh +# shebang is for editor file mode detection only + +function save_vars { + if [ -s $envfile ]; then + for var in $@; do + save_env --file $envfile $var + done + fi +} + +function save_chosen { + save_vars did_fai_check last_boot +} + +# fai_check is so we can act like a pxe boot, but just for fai, and by +# using /debian_bootstrap to do it. We toggle on and off the grub var +# did_fai_check so we can do the check every other boot. Then +# /debian_bootstrap checks for that var on boot and if we want to do a +# fai check, it does it, then reboots. But it also sets did_fai_check to +# a 3rd state os_true which means we did the fai check, and we don't +# want to do it again. This is useful for systems without libreboot. + +# We don't set this to fai check so we can't get into +# an infinite reboot cycle. We depend on the os to +# create the initial grubenv file. +set default=/debianstable_bootstrap # could use 0 here. +set timeout=1 + +for part in (ahci*4) (ata*4); do + envfile=$part/grubenv + if [ -s $envfile ]; then + load_env --file $envfile + if [ x$did_fai_check == xfalse -a x$last_boot != x$default ]; then + set default=fai-check + elif [ ! -z $last_boot ]; then + set default=$last_boot + fi + break + fi +done + +did_fai_check=false + +bs_dir=/debianstable_bootstrap +menuentry $bs_dir --id=$bs_dir { + # note, we might be able to use $chosen and avoid setting this here, + # and set it inside save_chosen. but I haven't tested it, + # it's just one less line of repitition. + last_boot=$1 + save_chosen + configfile $bs_dir/boot/grub/grub.cfg +} + +for dir in /boot_*; do + if [ $dir == '/boot_*' ]; then + break + fi + menuentry $dir --id=$dir { + last_boot=$1 + save_chosen + configfile $1/grub/grub.cfg + } +done + +menuentry fai-check --id=fai-check { + did_fai_check=true + save_vars did_fai_check + configfile $bs_dir/boot/grub/grub.cfg +} diff --git a/fai/config/files/boot/chboot/DEFAULT b/fai/config/files/boot/chboot/DEFAULT new file mode 100755 index 0000000..8a093d9 --- /dev/null +++ b/fai/config/files/boot/chboot/DEFAULT @@ -0,0 +1,129 @@ +#!/bin/bash +# Copyright (C) 2016 Ian Kelling + +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License +# as published by the Free Software Foundation; either version 2 +# of the License, or (at your option) any later version. + +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. + +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. + + +x="$(readlink -f "$BASH_SOURCE")"; source "${x%/*}/bash-trace" + +[[ $EUID == 0 ]] || exec sudo "$BASH_SOURCE" "$@" + +usage() { + cat </dev/null; then + mnt=/ +fi + +if [[ ! $distro ]]; then + echo "available distros:" + cur=$(btrfs subvol show $mnt| sed -rn 's/^.*Name:\s*(\S*).*/\1/p') + btrfs subvolume list $mnt | awk '{print $9}' | sed "s/$cur/$cur (current)/" + exit 0 +fi + +###### end command line parsing ##### + + +#### begin initial error checking ##### + +if ! btrfs subvolume list $mnt | grep "$distro$" &>/dev/null; then + echo "$0: error: $distro not found in btrfs subvolume list $mnt:" + btrfs subvolume list $mnt + exit 1 +fi + +#### end initial error checking ##### + +e() { echo "$@"; "$@"; } + +boot_dev=$(mount | sed -rn "s#^(\S+) on $mnt .*#\1#p") + +mount_point=$(mktemp -d) + +e mount -o subvol=$distro $boot_dev $mount_point + +boot_disk=${boot_dev%%[0-9]*} + +# arch doesn't have $mount_point/grub/device.map, accoring to the grub manual, +# it just generates one if the file doesn't exist. +# https://www.gnu.org/software/grub/manual/html_node/Device-map.html +e grub-bios-setup -d $mount_point/grub/i386-pc -s -m $mount_point/grub/device.map $boot_disk + +e umount $mount_point + +e mount $boot_disk$grub_extn $mount_point +e grub-editenv $mount_point/grubenv set last_boot=/$distro +e grub-editenv $mount_point/grubenv set did_fai_check=true +e umount $mount_point +e rmdir $mount_point + +if $reboot; then + touch /tmp/keyscript-off + reboot now +fi diff --git a/fai/config/files/etc/apt/preferences.d/belenos/BELENOS b/fai/config/files/etc/apt/preferences.d/belenos/BELENOS new file mode 100644 index 0000000..1e59c86 --- /dev/null +++ b/fai/config/files/etc/apt/preferences.d/belenos/BELENOS @@ -0,0 +1,3 @@ +Package: * +Pin: release a=belenos-backports +Pin-Priority: 500 diff --git a/fai/config/files/etc/apt/preferences.d/flidas/FLIDAS b/fai/config/files/etc/apt/preferences.d/flidas/FLIDAS new file mode 100644 index 0000000..bd19b25 --- /dev/null +++ b/fai/config/files/etc/apt/preferences.d/flidas/FLIDAS @@ -0,0 +1,3 @@ +Package: * +Pin: release a=flidas-backports +Pin-Priority: 500 diff --git a/fai/config/files/etc/apt/preferences.d/stable/STABLE b/fai/config/files/etc/apt/preferences.d/stable/STABLE new file mode 100644 index 0000000..662b957 --- /dev/null +++ b/fai/config/files/etc/apt/preferences.d/stable/STABLE @@ -0,0 +1,21 @@ +Explanation: tar, cuz https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=819978 +Explanation: kernel & btrfs-tools, because btrfs is getting a lot of active +Explanation: dev, and their mailing list says better to use recent version +Explanation: to avoid bugs. linux-base is needed for the kernel, +Explanation: which you can find out by failing +Explanation: apt-get install linux-image-amd64/jessie-backports +Explanation: And then trying aptitude -s install, or +Explanation: apt-get -t jessie-backports install linux-image-amd64 +Explanation: +Explanation: +Package: tar linux-image-amd64 linux-base btrfs-tools +Pin: release a=jessie-backports +Pin-Priority: 500 + +Package: * +Pin: release a=testing +Pin-Priority: -10 + +Package: * +Pin: release a=testing-updates +Pin-Priority: -10 diff --git a/fai/config/files/etc/apt/preferences.d/unstable/DEBIAN b/fai/config/files/etc/apt/preferences.d/unstable/DEBIAN new file mode 100644 index 0000000..87d6c00 --- /dev/null +++ b/fai/config/files/etc/apt/preferences.d/unstable/DEBIAN @@ -0,0 +1,19 @@ +Explanation: https://debian-handbook.info/browse/stable/sect.apt-get.html#sect.apt.priorities +Explanation: And man apt_preferences +Explanation: Installed packages get 100 priority, so this won't upgrade testing +Explanation: packages unless explicitly asked to. +Explanation: Less than 0 won't install package unless you specify the archive. +Explanation: This is good, so you never just search for a package and install +Explanation: it without knowing it's from a different archive. +Explanation: Install with apt-get install package/testing. But if dependencies are +Explanation: needed, or need upgrading, +Explanation: apt-get -t testing package, setting testing to priority +Explanation: 990 just for that command. +Explanation: Use apt-cache policy to verify these settings. +Package: * +Pin: release a=unstable +Pin-Priority: -20 + +Package: * +Pin: release a=unstable-updates +Pin-Priority: -20 diff --git a/fai/config/files/etc/apt/sources.list.d/belenos.list/BELENOS b/fai/config/files/etc/apt/sources.list.d/belenos.list/BELENOS new file mode 100644 index 0000000..17b28dc --- /dev/null +++ b/fai/config/files/etc/apt/sources.list.d/belenos.list/BELENOS @@ -0,0 +1,12 @@ +deb http://mirror.fsf.org/trisquel/ belenos main +deb-src http://mirror.fsf.org/trisquel/ belenos main + +deb http://mirror.fsf.org/trisquel/ belenos-updates main +deb-src http://mirror.fsf.org/trisquel/ belenos-updates main + +deb http://mirror.fsf.org/trisquel/ belenos-security main +deb-src http://mirror.fsf.org/trisquel/ belenos-security main + +# Uncomment this lines to enable the backports optional repository +deb http://mirror.fsf.org/trisquel/ belenos-backports main +deb-src http://mirror.fsf.org/trisquel/ belenos-backports main diff --git a/fai/config/files/etc/apt/sources.list.d/flidas.list/FLIDAS b/fai/config/files/etc/apt/sources.list.d/flidas.list/FLIDAS new file mode 100644 index 0000000..faec0da --- /dev/null +++ b/fai/config/files/etc/apt/sources.list.d/flidas.list/FLIDAS @@ -0,0 +1,12 @@ +deb http://mirror.fsf.org/trisquel/ flidas main +deb-src http://mirror.fsf.org/trisquel/ flidas main + +deb http://mirror.fsf.org/trisquel/ flidas-updates main +deb-src http://mirror.fsf.org/trisquel/ flidas-updates main + +deb http://mirror.fsf.org/trisquel/ flidas-security main +deb-src http://mirror.fsf.org/trisquel/ flidas-security main + +# Uncomment this lines to enable the backports optional repository +deb http://mirror.fsf.org/trisquel/ flidas-backports main +deb-src http://mirror.fsf.org/trisquel/ flidas-backports main diff --git a/fai/config/files/etc/apt/sources.list.d/stable.list/STABLE b/fai/config/files/etc/apt/sources.list.d/stable.list/STABLE new file mode 100644 index 0000000..0093b4f --- /dev/null +++ b/fai/config/files/etc/apt/sources.list.d/stable.list/STABLE @@ -0,0 +1,11 @@ +deb http://http.us.debian.org/debian jessie main +deb-src http://http.us.debian.org/debian jessie main + +deb http://security.debian.org/ jessie/updates main +deb-src http://security.debian.org/ jessie/updates main + +deb http://http.us.debian.org/debian jessie-updates main +deb-src http://http.us.debian.org/debian jessie-updates main + +deb http://http.debian.net/debian jessie-backports main +deb-src http://http.debian.net/debian jessie-backports main diff --git a/fai/config/files/etc/apt/sources.list.d/stable.list/STABLE_LINODE b/fai/config/files/etc/apt/sources.list.d/stable.list/STABLE_LINODE new file mode 100644 index 0000000..3e6e183 --- /dev/null +++ b/fai/config/files/etc/apt/sources.list.d/stable.list/STABLE_LINODE @@ -0,0 +1,24 @@ +deb http://mirrors.linode.com/debian/ jessie main +deb-src http://mirrors.linode.com/debian/ jessie main + +deb http://security.debian.org/ jessie/updates main +deb-src http://security.debian.org/ jessie/updates main + +# jessie-updates, previously known as 'volatile' +deb http://mirrors.linode.com/debian/ jessie-updates main +deb-src http://mirrors.linode.com/debian/ jessie-updates main + +deb http://mirrors.linode.com/debian/ jessie-backports main +deb-src http://mirrors.linode.com/debian/ jessie-backports main + +deb http://mirrors.linode.com/debian testing main +deb-src http://mirrors.linode.com/debian testing main + +deb http://security.debian.org/ testing/updates main +deb-src http://security.debian.org/ testing/updates main + +deb http://mirrors.linode.com/debian testing-updates main +deb-src http://mirrors.linode.com/debian testing-updates main + +deb http://mirrors.linode.com/debian unstable main +deb-src http://mirrors.linode.com/debian unstable main diff --git a/fai/config/files/etc/apt/sources.list.d/stable.list/STABLE_NONFREE b/fai/config/files/etc/apt/sources.list.d/stable.list/STABLE_NONFREE new file mode 100644 index 0000000..d5cc0db --- /dev/null +++ b/fai/config/files/etc/apt/sources.list.d/stable.list/STABLE_NONFREE @@ -0,0 +1,11 @@ +deb http://http.us.debian.org/debian jessie main contrib non-free +deb-src http://http.us.debian.org/debian jessie main contrib non-free + +deb http://security.debian.org/ jessie/updates main contrib non-free +deb-src http://security.debian.org/ jessie/updates main contrib non-free + +deb http://http.us.debian.org/debian jessie-updates main contrib non-free +deb-src http://http.us.debian.org/debian jessie-updates main contrib non-free + +deb http://http.debian.net/debian jessie-backports main contrib non-free +deb-src http://http.debian.net/debian jessie-backports main contrib non-free diff --git a/fai/config/files/etc/apt/sources.list.d/testing.list/STABLE_FREE b/fai/config/files/etc/apt/sources.list.d/testing.list/STABLE_FREE new file mode 120000 index 0000000..586e320 --- /dev/null +++ b/fai/config/files/etc/apt/sources.list.d/testing.list/STABLE_FREE @@ -0,0 +1 @@ +TESTING_FREE \ No newline at end of file diff --git a/fai/config/files/etc/apt/sources.list.d/testing.list/STABLE_NONFREE b/fai/config/files/etc/apt/sources.list.d/testing.list/STABLE_NONFREE new file mode 120000 index 0000000..b277a35 --- /dev/null +++ b/fai/config/files/etc/apt/sources.list.d/testing.list/STABLE_NONFREE @@ -0,0 +1 @@ +TESTING_NONFREE \ No newline at end of file diff --git a/fai/config/files/etc/apt/sources.list.d/testing.list/TESTING_FREE b/fai/config/files/etc/apt/sources.list.d/testing.list/TESTING_FREE new file mode 100644 index 0000000..031bddf --- /dev/null +++ b/fai/config/files/etc/apt/sources.list.d/testing.list/TESTING_FREE @@ -0,0 +1,11 @@ +deb http://http.us.debian.org/debian testing main +deb-src http://http.us.debian.org/debian testing main + +deb http://security.debian.org/ testing/updates main +deb-src http://security.debian.org/ testing/updates main + +deb http://http.us.debian.org/debian testing-updates main +deb-src http://http.us.debian.org/debian testing-updates main + +deb http://http.us.debian.org/debian unstable main +deb-src http://http.us.debian.org/debian unstable main diff --git a/fai/config/files/etc/apt/sources.list.d/testing.list/TESTING_NONFREE b/fai/config/files/etc/apt/sources.list.d/testing.list/TESTING_NONFREE new file mode 100644 index 0000000..3b57312 --- /dev/null +++ b/fai/config/files/etc/apt/sources.list.d/testing.list/TESTING_NONFREE @@ -0,0 +1,11 @@ +deb http://http.us.debian.org/debian testing main contrib non-free +deb-src http://http.us.debian.org/debian testing main contrib non-free + +deb http://security.debian.org/ testing/updates main contrib non-free +deb-src http://security.debian.org/ testing/updates main contrib non-free + +deb http://http.us.debian.org/debian testing-updates main contrib non-free +deb-src http://http.us.debian.org/debian testing-updates main contrib non-free + +deb http://http.us.debian.org/debian unstable main contrib non-free +deb-src http://http.us.debian.org/debian unstable main contrib non-free diff --git a/fai/config/files/etc/apt/sources.list.d/xenial.list/XENIAL64 b/fai/config/files/etc/apt/sources.list.d/xenial.list/XENIAL64 new file mode 100644 index 0000000..452f266 --- /dev/null +++ b/fai/config/files/etc/apt/sources.list.d/xenial.list/XENIAL64 @@ -0,0 +1,11 @@ +###### Ubuntu Main Repos +deb http://us.archive.ubuntu.com/ubuntu/ xenial main universe +deb-src http://us.archive.ubuntu.com/ubuntu/ xenial main universe + +###### Ubuntu Update Repos +deb http://us.archive.ubuntu.com/ubuntu/ xenial-security main universe +deb http://us.archive.ubuntu.com/ubuntu/ xenial-updates main universe +deb http://us.archive.ubuntu.com/ubuntu/ xenial-backports main universe +deb-src http://us.archive.ubuntu.com/ubuntu/ xenial-security main universe +deb-src http://us.archive.ubuntu.com/ubuntu/ xenial-updates main universe +deb-src http://us.archive.ubuntu.com/ubuntu/ xenial-backports main universe diff --git a/fai/config/files/etc/apt/sources.list/DEFAULT b/fai/config/files/etc/apt/sources.list/DEFAULT new file mode 100644 index 0000000..0c5461c --- /dev/null +++ b/fai/config/files/etc/apt/sources.list/DEFAULT @@ -0,0 +1 @@ +# intentionaly left empty. only using sources.list.d diff --git a/fai/config/files/etc/apt/sources.list/GNOME b/fai/config/files/etc/apt/sources.list/GNOME deleted file mode 100644 index 3f8c4da..0000000 --- a/fai/config/files/etc/apt/sources.list/GNOME +++ /dev/null @@ -1,2 +0,0 @@ -deb http://httpredir.debian.org/debian stretch main contrib non-free -deb http://httpredir.debian.org/debian-security stretch/updates main contrib non-free diff --git a/fai/config/files/etc/motd/FAIBASE b/fai/config/files/etc/motd/FAIBASE deleted file mode 100644 index 4e5a967..0000000 --- a/fai/config/files/etc/motd/FAIBASE +++ /dev/null @@ -1,3 +0,0 @@ - - -Plan your installation, and FAI installs your plan. diff --git a/fai/config/files/etc/systemd/system/fai_check.service/VOL_STABLE_BOOTSTRAP b/fai/config/files/etc/systemd/system/fai_check.service/VOL_STABLE_BOOTSTRAP new file mode 100644 index 0000000..cbe4272 --- /dev/null +++ b/fai/config/files/etc/systemd/system/fai_check.service/VOL_STABLE_BOOTSTRAP @@ -0,0 +1,9 @@ +[Unit] +Description=check whether to kexec to fai, reboot, or do nothing + +[Service] +Type=oneshot +ExecStart=/root/fai-check + +[Install] +WantedBy=network.target diff --git a/fai/config/files/root/.ssh/authorized_keys/.gitignore b/fai/config/files/root/.ssh/authorized_keys/.gitignore new file mode 100644 index 0000000..c078f52 --- /dev/null +++ b/fai/config/files/root/.ssh/authorized_keys/.gitignore @@ -0,0 +1,3 @@ +# empty directory +* +!.gitignore diff --git a/fai/config/files/root/fai-check/VOL_STABLE_BOOTSTRAP b/fai/config/files/root/fai-check/VOL_STABLE_BOOTSTRAP new file mode 100755 index 0000000..b20e8e9 --- /dev/null +++ b/fai/config/files/root/fai-check/VOL_STABLE_BOOTSTRAP @@ -0,0 +1,89 @@ +#!/bin/bash + +set -eE -o pipefail +trap 'echo "$0:$LINENO:error: \"$BASH_COMMAND\" returned $?" >&2' ERR + + + +usage() { + cat < deadline )); then + echo "fai-check: hit $NETWORK_TIMOUT_SECS s tftp server timeout" + return 0 + fi + sleep 1 + done + m pxe-kexec -n --ignore-whitelist -l fai-generated faiserver ||: +} + +case $1 in + -f|--force) + try-kexec + exit + ;; +esac + +first=true +for dev in $(btrfs fi show / | sed -rn 's#^\s*devid\s.*\s([^0-9 ]+)\S+$#\1#p' \ + |sort); do + dev+=4 + mount $dev /mnt + if $first; then + if [[ -e /mnt/grubenv ]]; then + set -x + source <(grub-editenv /mnt/grubenv list) + set +x + fi + first=false + # we could just as well check if last_boot != /debianstable_boostrap + # the intent with this one is just a little clearer. + if [[ $did_fai_check == true ]]; then + grub-editenv /mnt/grubenv set did_fai_check=os_true + # our service does not wait for network-online.target, + # because it will wait for too long when we don't have a network + # connection. So, we wait for 10 seconds. + # ref: https://www.freedesktop.org/wiki/Software/systemd/NetworkTarget/ + try-kexec ||: + else + return 0 + fi + else + # we make sure there is only 1 grubenv, + # so grub can just find the first one, in whatever order + # if looks at them, which may not be the same as us. + # If the disk dies, we just lose the default boot option, + # we will have to do manual steps to replace it anyways. + rm -f /mnt/gruvenv + fi + umount /mnt +done + +# the check for last_boot is not needed afaik, just sanity check. +if [[ $did_fai_check == true && $last_boot != /debianstable_boostrap ]]; then + # no need to reboot if we actually want to boot into this os. + reboot +fi diff --git a/fai/config/hooks/extrbase.DEFAULT b/fai/config/hooks/extrbase.DEFAULT new file mode 100755 index 0000000..4efa226 --- /dev/null +++ b/fai/config/hooks/extrbase.DEFAULT @@ -0,0 +1,8 @@ +#!/bin/bash + +# exit for any vm which is not our test vm +if ifclass VM && ! ifclass demohost; then + exit 0 +fi + +#chattr -Rf +C /target diff --git a/fai/config/hooks/instsoft.DEFAULT b/fai/config/hooks/instsoft.DEFAULT new file mode 100755 index 0000000..6d7f4c0 --- /dev/null +++ b/fai/config/hooks/instsoft.DEFAULT @@ -0,0 +1,44 @@ +#!/bin/bash + +# exit for any vm which is not our test vm +if ifclass VM && ! ifclass demohost || ifclass VOL_STABLE_BOOTSTRAP; then + exit 0 +fi + +keyfile=/var/lib/fai/config/distro-install-common/luks/host-$HOSTNAME +f=$target/root/keyscript +cat > $f <$f <<'EOF' +#!/bin/sh +if ! [ -e /tmp/key ]; then + stty -echo + read pass + printf '%s' "$pass" > /tmp/key +fi +cat /tmp/key +EOF +chmod +x $f + + +# for hosts which don't have these data volumes, copy the specific +# files we need. +if ifclass demohost; then + files=(/var/lib/fai/config/distro-install-common/luks/host-demohost) +elif ifclass tp; then + files=(/var/lib/fai/config/distro-install-common/luks/host-{tp,demohost}) +fi +if [[ ${files[0]} ]]; then + d=$target/q/root/luks + mkdir -p $d + cp ${files[@]} $d + chmod -R o-rwx $d +fi diff --git a/fai/config/hooks/partition.DEFAULT b/fai/config/hooks/partition.DEFAULT new file mode 100755 index 0000000..94f6c25 --- /dev/null +++ b/fai/config/hooks/partition.DEFAULT @@ -0,0 +1,455 @@ +#!/bin/bash -x +# Copyright (C) 2016 Ian Kelling + +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License +# as published by the Free Software Foundation; either version 2 +# of the License, or (at your option) any later version. + +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. + +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. + +set -eE -o pipefail +trap 'echo "$0:$LINENO:error: \"$BASH_COMMAND\" returned $?" >&2' ERR + +# # fai's setup-storage won't do btrfs on luks, +# # so we do it ourself :) +# inspiration taken from files in fai-setup-storage package + + +skiptask partition || ! type skiptask # for running not in fai + +#### begin configuration + +rootn=1 +swapn=2 +bootn=3 +# ext partition so grub can write persistent variables, +# so it can do a one time boot. grub can't write to +# btrfs or any cow fs because it's more +# more complicated to do and they don't want to. +grub_extn=4 +# bios boot partition, +# https://wiki.archlinux.org/index.php/GRUB +bios_grubn=5 +lastn=$bios_grubn +# this is larger than needed for several /boot subvols, +# becuase I keep a minimal debian install on it, for +# recovery needs, and for doing pxe-kexec. +boot_mib=10000 + + +##### end configuration + + +add-part() { # add partition suffix to $dev + local d ret + if [[ $# == 1 ]]; then + d=$dev + part=$1 + else + d=$1 + part=$2 + fi + if [[ $d == /dev/disk/by-id/* ]]; then + ret=$d-part$part + else + ret=$d$part + fi + echo $ret +} + +bootdev() { add-part $@ $bootn; } +rootdev() { add-part $@ $rootn; } +swapdev() { add-part $@ $swapn; } +grub_extdev() { add-part $@ $grub_extn; } +bios_grubdev() { add-part $@ $bios_grubn; } + +crypt-dev() { echo /dev/mapper/crypt_dev_${1##*/}; } +crypt-name() { echo crypt_dev_${1##*/}; } +root-cryptdev() { crypt-dev $(rootdev $@); } +swap-cryptdev() { crypt-dev $(swapdev $@); } +root-cryptname() { crypt-name $(rootdev $@); } +swap-cryptname() { crypt-name $(swapdev $@); } +devbyid() { + local f + for f in $FAI/distro-install-common/devbyid \ + /a/bin/fai/fai/config/distro-install-common/devbyid; do + if [[ -e $f ]]; then $f "$@"; fi + done + +} + +##### end function defs + +if ifclass REPARTITION;then + partition=true # force a full wipe +else + partition=false # change to true to force a full wipe +fi + + + +hdds=() +ssds=() +cd /sys/block +for disk in [sv]d[a-z]; do + case $(cat $disk/queue/rotational) in + 0) ssds+=(/dev/$disk) ;; + 1) hdds+=(/dev/$disk) ;; + *) echo "$0: error: unknown /sys/block/$disk/queue/rotational: \ +$(cat $disk/queue/rotational)"; exit 1 ;; + esac +done + +# install all ssds, or if there are none, all hdds +if ! ifclass ROTATIONAL && (( ${#ssds[@]} > 0 )); then + short_devs=( ${ssds[@]} ) +else + short_devs=( ${hdds[@]} ) +fi + +# check if the partitions exist have the right filesystems +#blkid="$(blkid -s TYPE)" +for dev in ${short_devs[@]}; do + if $partition; then break; fi + y=$(readlink -f $dev) + arr=($y[0-9]) + [[ ${#arr[@]} == "${lastn}" ]] || partition=true + for (( i=1; i <= lastn; i++ )); do + [[ -e ${dev}$i ]] || partition=true + done + # On one system, blkid is missing some partitions. + # maybe we need a flag, like FUZZY_BLKID or something, so we + # can check that at least some exist. + # for x in "`rootdev`: TYPE=\"crypto_LUKS\"" "`bootdev`: TYPE=\"btrfs\""; do + # echo "$blkid" | grep -Fx "$x" &>/dev/null || partition=true + # done +done + +if $partition && ifclass PARTITION_PROMPT; then + echo "Press any key except ctrl-c to continue and partition these drives:" + echo " ${short_devs[*]}" + read -r +fi + +devs=() +shopt -s extglob +for short_dev in ${short_devs[@]}; do + devs+=($(devbyid $short_dev)) +done + + +first=false +boot_devs=() +for dev in ${devs[@]}; do + if ifclass frodo; then + # I ran into a machine where the bios doesn't know about some disks, + # so 1st stage of grub also doesn't know about them. + # Also, grub does not support mounting degraded btrfs as far as + # I can tell with some googling. + # From within an arch install env, I could detect them by noting + # their partitions were mixed with the next disk in /dev/disk/by-path, + # and I have mixed model disks, and I could see the 8 models which showed + # up in the bios, and thus see which 2 models were missing. + # hdparm -I /dev/sdh will give model info in linux. + # However, in fai on jessie, /dev/disk/by-path dir doesn't exist, + # and I don't see another way, so I'm hardcoding them. + # We still put grub on them and partition them the same, for uniformity + # and in case they get moved to a system that can recognize them, + # we just exclude them from the boot filesystem. + cd /dev/disk/by-id/ + bad_disk=false + for id in ata-TOSHIBA_MD04ACA500_8539K4TQFS9A \ + ata-TOSHIBA_MD04ACA500_Y5IFK6IJFS9A; do + if [[ $(readlink -f $id) == "$(readlink -f $dev)" ]]; then + bad_disk=true + break + fi + done + $bad_disk || boot_devs+=(`bootdev`) + else + boot_devs+=(`bootdev`) + fi + if [[ $boot_devs && $first ]]; then + first_grub_extdev=`grub_extdev` + first=false + fi +done + +if ifclass RAID0 || (( ${#boot_devs[@]} < 4 )); then + raid_level=0 +else + raid_level=10 + # need double the space if we are raid 10, and then + # might as well give some extra. + boot_mib=$((boot_mib * 3)) +fi + + + +if [[ ! $DISTRO ]]; then + if ifclass VOL_STABLE_BOOTSTRAP; then + DISTRO=debianstable_bootstrap + elif ifclass VOL_STRETCH; then + DISTRO=debiantesting + elif ifclass VOL_STABLE; then + DISTRO=debianstable + elif ifclass VOL_XENIAL; then + DISTRO=ubuntuxenial + elif ifclass VOL_BELENOS; then + DISTRO=trisquelbelenos + elif ifclass VOL_FLIDAS; then + DISTRO=trisquelflidas + else + echo "PARTITIONER ERROR: no distro class/var set" >&2 + exit 1 + fi +fi +first_boot_dev=${boot_devs[0]} + + +bpart() { # btrfs a partition + case $raid_level in + 0) mkfs.btrfs -f $@ ;; + 10) mkfs.btrfs -f -m raid10 -d raid10 $@ ;; + esac +} + + +# keyfiles generated like: +# head -c 2048 /dev/urandom | od | s dd of=/q/root/luks/host-demohost +luks_dir=${LUKS_DIR:-/var/lib/fai/config/distro-install-common/luks} + +if [[ ! -e $luks_dir/host-$HOSTNAME ]]; then + echo "$0: error: no key for hostname at $luks_dir/host-$HOSTNAME" >&2 + exit 1 +fi + +if ifclass tp; then + lukspw=$(cat $luks_dir/traci) +else + lukspw=$(cat $luks_dir/iank) +fi +if ifclass demohost; then + lukspw=x +fi + + +first_root_crypt=$(root-cryptdev ${devs[0]}) + +# 1.5 x based on https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Installation_Guide/sect-disk-partitioning-setup-x86.html#sect-custom-partitioning-x86 +swap_mib=$(( $(grep ^MemTotal: /proc/meminfo | \ + awk '{print $2}') * 3/(${#devs[@]} * 2 ) / 1024 )) + +mkdir -p /tmp/fai +root_devs=() +for dev in ${devs[@]}; do + root_devs+=(`rootdev`) +done +shopt -s nullglob +if $partition; then + for dev in ${devs[@]}; do + # if we repartition to the same as an old partition, + # we don't want any old fses hanging around. + for (( i=1; i <= lastn; i++ )); do + x=$(add-part $dev $i) + [[ -e $x ]] || continue + count_down=10 + # wipefs has failed, manual run works, google suggests timing issue + while ! wipefs -a $x; do + sleep 2 + count_down=$((count_down - 1)) + (( count_down > 0 )) || exit 1 + done + done + done + for dev in ${devs[@]}; do + # parted will round up the disk size. Do -1 so we can have + # fully 1MiB unit partitions for easy resizing of the last partition. + # Otherwise we would pass in -0 for the end argument for the last partition. + # + # parted print error output is expected. example: + # Error: /dev/vda: unrecognised disk label + disk_mib=$(( $(parted -m $dev unit MiB print | \ + sed -nr "s#^/dev/[^:]+:([0-9]+).*#\1#p") - 1)) + root_end=$(( disk_mib - swap_mib - boot_mib / ${#boot_devs[@]} )) + swap_end=$(( root_end + swap_mib)) + + parted -s $dev mklabel gpt + # MiB because parted complains about alignment otherwise. + pcmd="parted -a optimal -s -- $dev" + $pcmd mkpart primary "ext3" 12MiB ${root_end}MiB + $pcmd mkpart primary "linux-swap" ${root_end}MiB ${swap_end}MiB + $pcmd mkpart primary "" ${swap_end}MiB ${disk_mib}MiB + # i only need a few k, but googling min size, + # I found someone saying that gparted required + # required at least 8 because of their hard drive cylinder size. + # And 8 is still very tiny. + $pcmd mkpart primary "ext2" 4MiB 12MiB + # gpt ubuntu cloud image uses ~4 mb for this partition. fai uses 1 MiB. + # so, I use 3, whatever. + # note: parted manual saying cheap flash media + # should to start at 4. + $pcmd mkpart primary "" 1MiB 4MiB + $pcmd set $bios_grubn bios_grub on + $pcmd set $bootn boot on # generally not needed on modern systems + # the mkfs failed before on a vm, which prompted me to add + # sleep .1 + # then it failed again on a physical machine + # with: + # Device /dev/disk/by-id/foo doesn't exist or access denied, + # so I added a wait until it existed. + # Then I added the mkfs.ext2, which claimed to succeed, + # but then couldn't be found upon reboot. In that case we didn't + # wait at all. So I've added a 3 second minimum wait. + sleep 3 + secs=0 + while [[ ! -e `rootdev` ]] && (( secs < 10 )); do + sleep 1 + secs=$((secs +1)) + done + # Holds just a single file, rarely written, so + # use ext2, like was often used for the /boot partition. + # This exists because grub can only persist data to a non-cow fs. + # And we use persisting a var in grub to do a one time boot. + # We could pass the data on the kernel command line and persist it + # to grubenv after booting, but that relies on the boot always succeeding. + # This is just a bit more robust, and it could work for booting + # into ipxe which can't persist data, if we ever got that working. + mkfs.ext2 `grub_extdev` + yes YES | cryptsetup luksFormat `rootdev` $luks_dir/host-$HOSTNAME \ + -c aes-cbc-essiv:sha256 -s 256 || [[ $? == 141 ]] + yes "$lukspw" | \ + cryptsetup luksAddKey --key-file $luks_dir/host-$HOSTNAME \ + `rootdev` || [[ $? == 141 ]] + # background: Keyfile and password are treated just + # like 2 ways to input a passphrase, so we don't actually need to have + # different contents of keyfile and passphrase, but it makes some + # security sense to a really big randomly generated passphrase + # as much as possible, so we have both. + # + # This would remove the keyfile. + # yes 'test' | cryptsetup luksRemoveKey /dev/... \ + # /key/file || [[ $? == 141 ]] + + cryptsetup luksOpen `rootdev` `root-cryptname` \ + --key-file $luks_dir/host-$HOSTNAME + done + ls -la /dev/btrfs-control # this was probably for debugging... + sleep 1 + bpart $(for dev in ${devs[@]}; do root-cryptdev; done) + bpart ${boot_devs[@]} +else + for dev in ${devs[@]}; do + mkfs.ext2 `grub_extdev` + cryptsetup luksOpen `rootdev` `root-cryptname` \ + --key-file $luks_dir/host-$HOSTNAME + done + sleep 1 +fi + + +if [[ $DISTRO != debianstable_bootstrap ]]; then + # bootstrap distro doesn't use separate encrypted root. + mount -o subvolid=0 $first_root_crypt /mnt + # systemd creates subvolumes we want to delete. + s=($(btrfs subvolume list --sort=-path /mnt | + sed -rn "s#^.*path\s*(root_$DISTRO/\S+)\s*\$#\1#p")) + for subvol in ${s[@]}; do btrfs subvolume delete /mnt/$subvol; done + btrfs subvolume set-default 0 /mnt + [[ ! -e /mnt/root_$DISTRO ]] || btrfs subvolume delete /mnt/root_$DISTRO + + ## create subvols ## + cd /mnt + + btrfs subvolume create root_$DISTRO + + mkdir -p /mnt/root_$DISTRO/boot + # could set default subvol like this, but no reason to. + # btrfs subvolume set-default \ + # $(btrfs subvolume list . | grep "root_$DISTRO$" | awk '{print $2}') . + + # no cow on the root filesystem. it's setup is fully scripted, + # if it's messed up, we will just recreated it, + # and we can get better perf with this. + # I can't remember exactly why, but this is preferable to mounting with + # -o nodatacow, I think because subvolumes inherit that. + chattr -Rf +C root_$DISTRO + cd / + umount /mnt +fi + +mount -o subvolid=0 $first_boot_dev /mnt +cd /mnt +btrfs subvolume set-default 0 /mnt # already default, just ensuring it. + +# for libreboot systems. grub2 only reads from subvolid=0 +mkdir -p /mnt/grub2 +cp $FAI/distro-install-common/libreboot_grub.cfg /mnt/grub2 + +if [[ $DISTRO == debianstable_bootstrap ]]; then + # this is just convenience for the libreboot_grub config + # so we can glob the other ones easier. + boot_vol=$DISTRO +else + boot_vol=boot_$DISTRO +fi +[[ ! -e /mnt/$boot_vol ]] || btrfs subvolume delete /mnt/$boot_vol +btrfs subvolume create $boot_vol +cd / +umount /mnt +## end create subvols ## + +dev=${boot_devs[0]} +mount $first_grub_extdev /mnt +grub-editenv /mnt/grubenv set did_fai_check=true +grub-editenv /mnt/grubenv set last_boot=/$boot_vol +umount /mnt + +if [[ $DISTRO == debianstable_bootstrap ]]; then + cat > /tmp/fai/fstab </tmp/fai/disk_var.sh < /tmp/fai/fstab <>/tmp/fai/crypttab <> /tmp/fai/fstab </tmp/fai/disk_var.sh < $target/etc/apt/sources.list -# external mirror -deb MIRRORURL $ubuntudist main restricted universe multiverse -deb MIRRORURL $ubuntudist-updates main restricted universe multiverse -deb MIRRORURL $ubuntudist-security main restricted universe multiverse -EOM - -# determine a fast mirror for Ubuntu -list=$(curl -s http://mirrors.ubuntu.com/mirrors.txt) -mirror=$(netselect $list | awk '{print $2}') -sed -i -e "s#MIRRORURL#$mirror#" $target/etc/apt/sources.list diff --git a/fai/config/hooks/updatebase.UBUNTU b/fai/config/hooks/updatebase.UBUNTU new file mode 120000 index 0000000..77bc497 --- /dev/null +++ b/fai/config/hooks/updatebase.UBUNTU @@ -0,0 +1 @@ +updatebase.DEBIAN \ No newline at end of file diff --git a/fai/config/package_config/DEBIAN b/fai/config/package_config/DEBIAN index bdec0d6..d3dc95e 100644 --- a/fai/config/package_config/DEBIAN +++ b/fai/config/package_config/DEBIAN @@ -13,11 +13,13 @@ memtest86+ PACKAGES install DHCPC isc-dhcp-client +# ian: note everything after the grub package should be refactored into +# a new class. PACKAGES install GRUB_PC -grub-pc +grub-pc cryptsetup btrfs-tools sudo bridge-utils netcat-openbsd PACKAGES install GRUB_EFI -grub-efi +grub-efi cryptsetup btrfs-tools sudo bridge-utils netcat-openbsd PACKAGES install LVM lvm2 diff --git a/fai/config/package_config/FAIBASE b/fai/config/package_config/FAIBASE index e4851b1..9888175 100644 --- a/fai/config/package_config/FAIBASE +++ b/fai/config/package_config/FAIBASE @@ -1,4 +1,4 @@ -PACKAGES install-norec DEBIAN +PACKAGES install-norec DEBIAN UBUNTU fai-client debconf-utils file diff --git a/fai/config/package_config/UBUNTU b/fai/config/package_config/UBUNTU index afd01e7..1c7f8e0 100644 --- a/fai/config/package_config/UBUNTU +++ b/fai/config/package_config/UBUNTU @@ -13,7 +13,7 @@ PACKAGES install DHCPC isc-dhcp-client PACKAGES install GRUB_PC -grub-pc +grub-pc cryptsetup btrfs-tools bridge-utils netcat-openbsd PACKAGES install XORG ubuntu-desktop diff --git a/fai/config/scripts/DEBIAN/20-capabilities b/fai/config/scripts/DEBIAN/20-capabilities index ea650fa..b057587 100755 --- a/fai/config/scripts/DEBIAN/20-capabilities +++ b/fai/config/scripts/DEBIAN/20-capabilities @@ -4,6 +4,9 @@ # Restore them here. # +# note on an ubuntu 16.04 system, these caps were set without this script +# running. I wonder if it is actually needed on a debian 8 system. + set -e if [ ! -x $target/sbin/setcap ] ; then diff --git a/fai/config/scripts/DEBIAN/30-interface b/fai/config/scripts/DEBIAN/30-interface index 924b236..9229207 100755 --- a/fai/config/scripts/DEBIAN/30-interface +++ b/fai/config/scripts/DEBIAN/30-interface @@ -1,8 +1,8 @@ #! /bin/bash +# modified from upstream fai example error=0; trap 'error=$(($?>$error?$?:$error))' ERR # save maximum error code - newnicnames() { # determine predictable network names only for stretch and above @@ -24,6 +24,10 @@ newnicnames() { break fi done + # This condition is only needed because the nfsroot I use + # is based on Jessie, which has an old udev which can't + # figure out the persistent interface name used in stretch. + if ifclass VM; then NIC1=ens3; return; fi if [[ ! $name ]]; then echo "$0: error: could not find systemd predictable network name. Using $NIC1." fi @@ -33,14 +37,34 @@ newnicnames CIDR=$(ip -o -f inet addr show $NIC1 | awk '{print $4}') if ifclass DHCPC && [ $FAI_ACTION = "install" -o $FAI_ACTION = "dirinstall" ]; then - cat > $target/etc/network/interfaces <<-EOF + if ifclass VM; then + # note, this condition would apply to the elif below too, + # but I don't specify a static ip in fai, so not bothering + cat > $target/etc/network/interfaces <<-EOF # generated by FAI auto lo $NIC1 iface lo inet loopback iface $NIC1 inet dhcp EOF + else + cat > $target/etc/network/interfaces <<-EOF + # generated by FAI + auto lo br0 + iface lo inet loopback + iface $NIC1 inet manual + # make a bridge by default so we can have bridged vms. + # Some example I read had stp on, but i don't need stp, + # and it causes a vm to fail pxe boot, presumably unless + # you add some delay. + # http://wiki.libvirt.org/page/PXE_boot_%28or_dhcp%29_on_guest_failed + iface br0 inet dhcp + bridge_ports $NIC1 + bridge_stp off + bridge_maxwait 0 +EOF + fi elif [ $FAI_ACTION = "install" -o $FAI_ACTION = "dirinstall" ]; then - [ -n "$CIDR" ] && cat > $target/etc/network/interfaces <<-EOF + [ -n "$CIDR" ] && cat > $target/etc/network/interfaces <<-EOF # generated by FAI auto lo $NIC1 iface lo inet loopback diff --git a/fai/config/scripts/DEBIAN/40-misc b/fai/config/scripts/DEBIAN/40-misc index 8308bbe..8d4e85d 100755 --- a/fai/config/scripts/DEBIAN/40-misc +++ b/fai/config/scripts/DEBIAN/40-misc @@ -3,6 +3,22 @@ # (c) Thomas Lange, 2001-2016, lange@debian.org # (c) Michael Goetze, 2010-2011, mgoetze@mgoetze.net + +# on ubuntu 16.04 which didn't run this script, some things which didn't +# apply: +# /etc/dpkg/dpkg.cfg.d/fai didn't exist, +# machine-id was already setup. + +# on that system and a debian stretch system, after reboot, +# some things done here don't seem to persist: +# some thin/etc/mtab is symlink somewhere else, +# and mailname is $HOSTNAME.lan + +# the adjtime thing is to support changing the system clock +# from representing UTC (the default) to localtime (windows default). + +# afaik, the only useful thing here for me is setting /etc/hostname + error=0; trap 'error=$(($?>$error?$?:$error))' ERR # save maximum error code # a list of modules which are loaded at boot time diff --git a/fai/config/scripts/DEMO/10-misc b/fai/config/scripts/DEMO/10-misc deleted file mode 100755 index 45f0573..0000000 --- a/fai/config/scripts/DEMO/10-misc +++ /dev/null @@ -1,24 +0,0 @@ -#! /bin/bash - -# (c) Thomas Lange, 2001-2013, lange@debian.org - -error=0; trap 'error=$(($?>$error?$?:$error))' ERR # save maximum error code - -ifclass XORG && { - fcopy -M /etc/X11/xorg.conf -} - -if ifclass UBUNTU; then - groups="adm cdrom sudo dip plugdev lpadmin sambashare" - $ROOTCMD addgroup --system lpadmin || true - $ROOTCMD addgroup --system sambashare || true -fi - -# add a demo user account -if ! $ROOTCMD getent passwd demo ; then - $ROOTCMD adduser --disabled-login --gecos "fai demo user" demo - $ROOTCMD usermod -p "$ROOTPW" demo - for g in $groups; do - $ROOTCMD adduser demo $g - done -fi diff --git a/fai/config/scripts/GRUB_PC/10-setup b/fai/config/scripts/GRUB_PC/10-setup index 7a04760..160dfa2 100755 --- a/fai/config/scripts/GRUB_PC/10-setup +++ b/fai/config/scripts/GRUB_PC/10-setup @@ -26,11 +26,11 @@ if [ $FAI_ACTION != "install" ]; then fi $ROOTCMD grub-mkdevicemap --no-floppy -GROOT=$($ROOTCMD grub-probe -tdrive -d $BOOT_DEVICE) # Check if RAID is used for the boot device if [[ $BOOT_DEVICE =~ '/dev/md' ]]; then + GROOT=$($ROOTCMD grub-probe -tdrive -d $BOOT_DEVICE) raiddev=${BOOT_DEVICE#/dev/} # install grub on all members of RAID for device in `LC_ALL=C perl -ne 'if(/^'$raiddev'\s.+raid\d+\s(.+)/){ $_=$1; s/\d+\[\d+\]//g; print }' /proc/mdstat`; do @@ -49,10 +49,13 @@ EOF rm $target/boot/grub/device.map else - $ROOTCMD grub-install --no-floppy "$GROOT" - if [ $? -eq 0 ]; then - echo "Grub installed on $BOOT_DEVICE = $GROOT" - fi + for dev in $BOOT_DEVICE; do + GROOT=$($ROOTCMD grub-probe -tdrive -d $dev) + $ROOTCMD grub-install --no-floppy "$GROOT" + if [ $? -eq 0 ]; then + echo "Grub installed on $dev = $GROOT" + fi + done fi $ROOTCMD update-grub diff --git a/fai/config/scripts/GRUB_PC/11-ian b/fai/config/scripts/GRUB_PC/11-ian new file mode 100755 index 0000000..72dc7ec --- /dev/null +++ b/fai/config/scripts/GRUB_PC/11-ian @@ -0,0 +1,100 @@ +#!/bin/bash -x + +set -eE -o pipefail +trap 'echo "$0:$LINENO:error: \"$BASH_COMMAND\" returned $?" >&2' ERR + +if [[ $EUID != 0 ]]; then + echo "$0: error: expected to be root." + exit 1 +fi + +if ! type -t fcopy &>/dev/null; then + sudo apt-get -y install fai-client +fi + +chroot $FAI_ROOT bash <<'EOFOUTER' +if getent group systemd-journal >/dev/null; then + # makes the journal be saved to disk. + mkdir -p /var/log/journal + chmod 755 /var/log/journal +fi +debconf-set-selections </dev/null; then + usermod -aG systemd-journal iank +fi +# https://askubuntu.com/questions/33416/how-do-i-disable-the-boot-splash-screen-and-only-show-kernel-and-boot-text-inst +# it suggests not having plymouth-theme-ubuntu-text, but +# making it not installed then kills plymouth, then makes +# the system not boot. +sed -ri 's/(^ *GRUB_CMDLINE_LINUX.*)quiet splash/\1/' /etc/default/grub +# on xenial, no grub is displayed at all. fix that. +# found just by noticing this in the config file, and a +# warning about it in error.log +sed -i '/^ *GRUB_HIDDEN_TIMEOUT/d' /etc/default/grub +update-grub2 +EOF + + +# reading through the groups that iank is in but traci isn't, +for g in plugdev audio video cdrom; do + $ROOTCMD usermod -a -G $g traci +done diff --git a/fai/config/scripts/LAST/50-misc b/fai/config/scripts/LAST/50-misc index aa4c198..4627697 100755 --- a/fai/config/scripts/LAST/50-misc +++ b/fai/config/scripts/LAST/50-misc @@ -45,7 +45,7 @@ if [ $do_init_tasks -eq 1 ] ; then fi # Make sure everything is configured properly -if ifclass DEBIAN ; then +if ifclass DEBIAN || ifclass UBUNTU; then echo "Running \"apt-get -f install\" for the last time." $ROOTCMD apt-get -f install fi diff --git a/fai/config/scripts/UBUNTU/10-rootpw b/fai/config/scripts/UBUNTU/10-rootpw new file mode 120000 index 0000000..c0d9817 --- /dev/null +++ b/fai/config/scripts/UBUNTU/10-rootpw @@ -0,0 +1 @@ +../DEBIAN/10-rootpw \ No newline at end of file diff --git a/fai/config/scripts/UBUNTU/30-interface b/fai/config/scripts/UBUNTU/30-interface new file mode 120000 index 0000000..e3dce43 --- /dev/null +++ b/fai/config/scripts/UBUNTU/30-interface @@ -0,0 +1 @@ +../DEBIAN/30-interface \ No newline at end of file diff --git a/fai/config/scripts/UBUNTU/40-misc b/fai/config/scripts/UBUNTU/40-misc new file mode 120000 index 0000000..8143e00 --- /dev/null +++ b/fai/config/scripts/UBUNTU/40-misc @@ -0,0 +1 @@ +../DEBIAN/40-misc \ No newline at end of file diff --git a/fai/config/scripts/UBUNTU/90-apt b/fai/config/scripts/UBUNTU/90-apt old mode 100755 new mode 100644 index f08a23d..5e2d1e3 --- a/fai/config/scripts/UBUNTU/90-apt +++ b/fai/config/scripts/UBUNTU/90-apt @@ -1,25 +1,10 @@ #! /bin/bash -if ifclass GERMAN; then - $ROOTCMD locale-gen LANG=de_DE.UTF-8 - $ROOTCMD update-locale LANG=de_DE.UTF-8 -else - ainsl -v /etc/locale.gen '^en_US.UTF-8 UTF-8' - $ROOTCMD locale-gen - $ROOTCMD update-locale LANG=en_US.UTF-8 -fi - -# check if we already use an external mirror -grep -q "external mirror" $target/etc/apt/sources.list && exit 0 - -cat < $target/etc/apt/sources.list -# external mirror -deb MIRRORURL $ubuntudist main restricted universe multiverse -deb MIRRORURL $ubuntudist-updates main restricted universe multiverse -deb MIRRORURL $ubuntudist-security main restricted universe multiverse -EOM - -# determine a fast mirror for Ubuntu -list=$(curl -s http://mirrors.ubuntu.com/mirrors.txt) -mirror=$(netselect $list | awk '{print $2}') -sed -i -e "s#MIRRORURL#$mirror#" $target/etc/apt/sources.list +# note: the name of this scripts doesn't fit it's contents, but it's +# because we are taking just part of the corresponding fai example +# script, and it's easier to keep up with upstream changes if the file +# name is the same. + +ainsl -v /etc/locale.gen '^en_US.UTF-8 UTF-8' +$ROOTCMD locale-gen +$ROOTCMD update-locale LANG=en_US.UTF-8 diff --git a/faiserver-disable b/faiserver-disable new file mode 100755 index 0000000..3cb6da2 --- /dev/null +++ b/faiserver-disable @@ -0,0 +1,26 @@ +#!/bin/bash + +set -eE -o pipefail +trap 'echo "$0:$LINENO:error: \"$BASH_COMMAND\" returned $?" >&2' ERR + +usage() { + cat </dev/null; then + sed -ri --follow-symlinks '\%^/srv/fai/d' /etc/exports + exportfs -ra +else + rm -f /etc/apache2/sites-enabled/faiserver.conf + systemctl reload apache2 +fi +EOF diff --git a/faiserver-revm b/faiserver-revm new file mode 100755 index 0000000..65a23d6 --- /dev/null +++ b/faiserver-revm @@ -0,0 +1,48 @@ +#!/bin/bash -l +set -x + +set -eE -o pipefail +cleanup() { :; }; _errcatch_cleanup=cleanup + +x="$(readlink -f "$BASH_SOURCE")"; source "${x%/*}/bash-trace" +cd ${x%/*} + +usage() { + cat </dev/null; then + apt-get install -y wget +fi + +armhf() { + [[ $(dpkg --print-architecture) == armhf ]] +} + +if grep -xFq 'VERSION="9 (stretch)"' /etc/os-release; then + # if we use stretch, no need for fai-project repo. + # this will need to be updated when there is a codename + # for stretch+1 + rm -f /etc/apt/sources.list.d/fai.list +elif armhf; then + if apt-cache policy | grep o=Debian,a=testing,n=stretch &>/dev/null; then + cat >/etc/apt/sources.list.d/testing.list <<'EOF' +deb http://http.us.debian.org/debian testing main contrib non-free +deb-src http://http.us.debian.org/debian testing main contrib non-free + +deb http://security.debian.org/ testing/updates main contrib non-free +deb-src http://security.debian.org/ testing/updates main contrib non-free + +deb http://http.us.debian.org/debian testing-updates main contrib non-free +deb-src http://http.us.debian.org/debian testing-updates main contrib non-free +EOF + + cat >/etc/apt/preferences.d/fai <<'EOF' +Package: fai-server fai-client fai-doc +Pin: release a=testing +Pin-Priority: 500 + +Package: * +Pin: release a=testing +Pin-Priority: -10 +EOF + fi +else + wget -O - http://fai-project.org/download/074BCDE4.asc | apt-key add - + cat >/etc/apt/sources.list.d/fai.list <<'EOF' +deb http://fai-project.org/download jessie koeln +EOF +fi + +# for ubuntu: +#add-apt-repository -y ppa:fai/ppa + +# for debian: + + +apt-get update + +# Relevant packages from fai-quickstart depends and fai-server recommends. +# I especially do not wait isc-dhcp-server or an inetd. Also excludes +# nfs-kernel-server. On an android chroot, we don\'t have nfs in the +# kernel, or the ability to install it. +pkgs=(fai-doc tftpd-hpa tar reprepro squashfs-tools binutils) +if modprobe nfsd &>/dev/null; then + pkgs+=(nfs-kernel-server) +else + pkgs+=(apache2) +fi + + +apt-get install -y ${pkgs[@]} +apt-get install --no-install-recommends -y fai-server + +r=http://http.us.debian.org/debian +# like default, but scrap httpredir, and nonfree. +# All my systems should be able to get along without nonfree +# for a base working system afaik. +cat >/etc/fai/apt/sources.list <>/etc/fai/apt/sources.list <<'EOF' +# uncommenting this from the defaults. it's got bug fixes. +# repository that may contain newer fai packages for jessie +deb http://fai-project.org/download jessie koeln +# fix tar https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=819978 +deb http://ftp.debian.org/debian jessie-backports main +EOF + + # note, fai doesn\'t look at /etc/fai/apt/preferences.d + cat >/etc/fai/apt/preferences <<'EOF' +Package: tar +Pin: release a=jessie-backports +Pin-Priority: 500 +EOF +fi + + +# tried out a stretch base, doesn't work yet. +# +$sed -f - /etc/fai/nfsroot.conf <>/srv/fai/nfsroot/root/.ssh/known_hosts + +# initially did the basic fai-chboot -Iv $std_arg default +# but found in console that it wanted to mount nfsroot +# to be the same as my dhcp server. +# Figured out to change the root= parameter from googling, +# and seeing fai-chboot -L +# using hostname failed. +# for -f, combined the 2 defaults so it will reboot and print to screen. + +# Add debug to -f flag for more verbose output. + + +# background on choosing apt-cacher-ng: +# googling around a bit finds 2 main solutions: +# http://askubuntu.com/questions/3503/best-way-to-cache-apt-downloads-on-a-lan +# apt-cacher-ng doesn\'t have zeroconf. +# It touts having minimal dependencies, but I don\'t care. +# The downside to squid-deb-proxy is that it\'s config is for specific repos, +# you have to add all the repos you use. +# That is the main reason I use apt-cacher-ng. +# It has a web portal, at http://faiserver:3142/acng-report.html + + +# random fai note: as far as I can tell, profiles are just for putting +# in a selectable boot menu, which I don\'t want. + +# the logsave prompted because the hostname faiserver was uknown. +# Here it was faiserver.lan when running from a faiserver vm. +# When running from a normal host with faiserver alias, it was the normal hosts name. +$sed 's/(^[^,]+,)\S+/\1faiserver/' /srv/fai/nfsroot/root/.ssh/known_hosts +# ditch the logo banner up top which screws with less. +touch /srv/fai/nfsroot/.nocolorlogo diff --git a/faiserver-uninstall b/faiserver-uninstall new file mode 100755 index 0000000..71a4ea0 --- /dev/null +++ b/faiserver-uninstall @@ -0,0 +1,36 @@ +#!/bin/bash +# Copyright (C) 2016 Ian Kelling + +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License +# as published by the Free Software Foundation; either version 2 +# of the License, or (at your option) any later version. + +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. + +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. + +set -eE -o pipefail +trap 'echo "$0:$LINENO:error: \"$BASH_COMMAND\" returned $?" >&2' ERR + +[[ $EUID == 0 ]] || exec sudo "${BASH_SOURCE}" "$@" + +usage() { + cat <. + +TODO: only tested on stretch. deactivation of swap on reboot +probably needs to be fixed on other oses. Even on stretch, +we get 1.5 minutes of waiting for the crypt_dev and normal +boot .device units. + +Warning!!! Backup your data. This script could have bugs in it. + +-n Dry run. Note, this likely won't be the exact commands, + for example, if you are running outside a vm, there will + probably be a reboot required in the middle so the kernel + can know about partition changes. +-r Reboot right away if it's needed. +-f Force running on a distro that has not been tested. +-h|--help Print help and exit. + +SIZE is MiB, or if g is specified, GiB. + +If using multiple devices, SIZE is applied to each device, so total change is +SIZE * devices. + +Note: Uses GNU getopt options parsing style +EOF + exit $1 +} + +reboot_not=false +dry_run=false +force=false + +temp=$(getopt -l help rnfh "$@") || usage 1 +eval set -- "$temp" +while true; do + case $1 in + -r) reboot_now=true; shift ;; + -n) dry_run=true; shift ;; + -f) force=true; shift ;; + -h|--help) usage ;; + --) shift; break ;; + *) echo "$0: Internal error!" ; exit 1 ;; + esac +done + +#### begin arg error checking #### + +if [[ $# != 2 ]]; then + echo "$0: error: expected 2 arguments" + usage 1 +fi + +if [[ $1 != [+-][0-9]* ]]; then + echo "$0: error: bad 1st arg: $1" + usage 1 +fi + +if ! which parted &>/dev/null; then + echo "$0: error: install parted" + exit 1 +fi + +case $2 in swap|boot) : ;; *) echo "$0: error: bad 2nd arg"; usage 1 ;; esac + +if ! $force && ! grep -q 'VERSION=.*stretch' /etc/os-release; then + echo "$0: error: This distro is untested. Only tested version atm is Stretch." + exit 1 +fi + +#### end arg error checking #### + + +boot=true +[[ $2 == boot ]] || boot=false + +op_size=$1 # operator plus size +if [[ $op_size == *g ]]; then + op_size=${op_size%g} + size=${op_size#?} + op=${op_size%$size} + size=$(( $size * 1024 )) + op_size=$op$size +else + size=${op_size#?} +fi + +if [[ $op_size == +* ]]; then + op_size_rev=-$size # rev = reverse + grow=true +else + op_size_rev=+$size + grow=false +fi + +##### end command line parsing ######## + +rootn=1 +swapn=2 +bootn=3 +needs_reboot=false +reboot_script_initialized=false + +pmk() { # partition make + part=$1 + start_op=$2 + end_op=$3 + fs_type="$4" + + # This fails outside a vm, but actually succeeds. also prints this + # message: + # Error: Partition(s) 2 on /dev/sda have been written, but + # we have been unable to inform the kernel of the change, probably + # because it/they are in use. As a result, the old partition(s) + # will remain in use. You should reboot now before making further + # changes. + + if ! p mkpart primary "$fs_type" \ + $((${ptable[start$part]} $start_op)) $((${ptable[end$part]} $end_op)); then + echo "$0: warning: ignoring failure return of mkpart" + fi +} + +def-e() { + if $dry_run; then + e() { echo "+ $@"; } + else + e() { echo "+ $@"; "$@"; } + fi +} + +def-e + + +while read devid dev; do + case $dev in + /dev/dm-[0-9]) + # older oses, it points to /dev/dm-x + dev=$(dmsetup info $dev | sed -rn 's/^\s*Name:\s*(\S*)/\1/p') + ;; + /dev/mapper/*) + dev=${dev#/dev/mapper/} + ;; + *) + echo "$0: error: could not find devicemapper root dev, +make sure you are running from a encrypted root this script is resizing" + exit 1 + ;; + esac + if [[ $dev != crypt_dev_*-part$rootn ]]; then + echo "$0: error: unexpected root device name, +make sure you are running from a encrypted root this script is resizing" + exit 1 + fi + dev=${dev#crypt_dev_} + dev=${dev%-part$rootn} + devpath=/dev/disk/by-id/$dev + echo skip=$size + def-e + declare -A ptable + while IFS=: read id start end psize _; do + [[ $id == [0-9] ]] || continue + ptable[start$id]=start=${start%%[^0-9]*} + ptable[end$id]=${end%%[^0-9]*} + ptable[size$id]=${psize%%[^0-9]*} + done < <(parted -m $devpath unit MiB print) + parted $devpath unit MiB print | tee /root/backup_partition_table_$dev + p() { e parted -a optimal -s -- $devpath unit MiB "$@"; } + unit=systemd-cryptsetup@crypt_dev_$dev-part$swapn + # note systemctl show can test if a unit exists. + if ! e systemctl stop $unit; then + e swapoff -a + fi + # there is a bug in jessie. this and the .swap unit are + # generated from /etc/fstab, and it escapes - to x2d, then doesn't escape it + # when looking for the file to use as swap. so, no swap is working on jessie + # right now. + sleep 1 # dunno if this is needed, + # but systemd likes to do these kind of things in the background. + + # These partition comments seems a little verbose now, but I bet they + # will be helpfull if I read this in more than a week from now. + # <> = deleted partition, () = partition + p rm $swapn # ( root )< swap >( boot ) + + root_resize_cmd="e btrfs fi resize $devid:${op_size_rev}M /" + if $grow; then $root_resize_cmd; fi + # if $grow; then + # < root >< swap >( boot ) + # ( root ) >< swap >( boot ) + # else + # < root >< swap >( boot ) + # ( root >< ) swap >( boot ) + + out=$(p rm $rootn 2>&1) + echo "$out" + + pmk $rootn "" $op_size_rev + + if echo "$out" | \ + grep "but we have been unable to inform the kernel" &>/dev/null; then + needs_reboot=true + fi + if $needs_reboot; then + # note: even if these units don't exist, this will succeed. + e systemctl mask dev-mapper-crypt_swap_$dev$swapn.swap + e systemctl mask systemd-cryptsetup@crypt_swap_$dev$swapn.service + e() { echo "$@" >> /root/finish-resize; } + if ! $reboot_script_initialized; then + reboot_script_initialized=true + rm -rf /root/finish-resize + cat >/root/finish-resize <<'EOF' +#!/bin/bash -x +set -eE -o pipefail +trap 'echo "$0:$LINENO:error: \"$BASH_COMMAND\" returned $?" >&2' ERR +EOF + chmod +x /root/finish-resize + fi + e swapoff -a + e systemctl unmask systemd-cryptsetup@crypt_swap_$dev$swapn.service + e systemctl unmask dev-mapper-crypt_swap_$dev$swapn.swap + fi + if ! $grow; then + $root_resize_cmd + fi + if $boot; then + # non by-id path, to match what btrfs fi show will tell us + boot_dev_path=$(readlink -f $devpath-part$bootn) + boot_devid=$(btrfs fi show /boot | \ + sed -rn "s#^\s*devid\s+(\S+)\s.*$boot_dev_path#\1#p") + + if ! $grow; then + # shrink boot, move it to a temp file + e btrfs fi resize $boot_devid:${op_size}M /boot + e umount /boot + temp_boot=/root/temp_boot_$dev + e dd bs=1M if=$boot_dev_path of=$temp_boot \ + count=$((${ptable[size$bootn]} $op_size)) + else + e umount /boot + fi + # if $grow; then + # ( root ) >< swap >< boot > + # ( root )( >< swap ) >< boot > + # ( root )( >< swap )( >< boot ) + # else + # ( root >< ) swap >< boot > + # ( root >< )( swap >< ) boot > + # ( root >< )( swap >< )( boot ) + p rm $bootn + pmk $swapn $op_size_rev $op_size_rev "linux-swap" + pmk $bootn $op_size_rev "" + + if $grow; then + e dd bs=1M if=$boot_dev_path of=$boot_dev_path skip=$size + e mount /boot + e btrfs fi resize $boot_devid:${op_size}M /boot + else + e dd bs=1M if=$temp_boot of=$boot_dev_path + e mount /boot + fi + else + # if $grow; then ( root )( >< swap )( boot ) + # else ( root >< )( swap )( boot ) + pmk $swapn $op_size_rev "" "linux-swap" + e systemctl start systemd-cryptsetup@crypt_swap_$dev$swapn + fi +done < <(btrfs fi show / | sed -nr 's#^\s*devid\s*(\S+).* path (.*)$#\1 \2#p') + + +if $boot; then + e rm -rf "/root/temp_boot_*" + e rm -f /root/finish-resize +fi + +if $needs_reboot; then + echo "$0: Reboot, run /root/finish-resize. It's contents:" + cat /root/finish-resize + if $reboot_now; then + echo "$0: rebooting now" + reboot now + exit + fi +fi diff --git a/install-chboot b/install-chboot new file mode 100755 index 0000000..098b232 --- /dev/null +++ b/install-chboot @@ -0,0 +1,54 @@ +#!/bin/bash +# Copyright (C) 2016 Ian Kelling + +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License +# as published by the Free Software Foundation; either version 2 +# of the License, or (at your option) any later version. + +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. + +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. + +[[ $EUID == 0 ]] || exec sudo -E "$BASH_SOURCE" "$@" + +x="$(readlink -f "$BASH_SOURCE")"; source "${x%/*}/bash-trace" +cd ${x%/*} + +usage() { + cat </dev/null || continue + if [[ -e $dir/boot ]]; then + dir=$dir/boot + fi + e install -m 755 -o root -g root chboot $dir +done +e umount $mount_point +e rmdir $mount_point diff --git a/live-kexec b/live-kexec new file mode 100644 index 0000000..5bd5833 --- /dev/null +++ b/live-kexec @@ -0,0 +1,59 @@ +#!/bin/bash +# Copyright (C) 2016 Ian Kelling + +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License +# as published by the Free Software Foundation; either version 2 +# of the License, or (at your option) any later version. + +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. + +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. + +usage() { + cat </dev/null; then + apt-get update + apt-get install -y debconf + debconf-set-selections <&2' ERR + +x=$(readlink -f "$BASH_SOURCE"); cd ${x%/*} + +usage() { + cat <&2' ERR + + +fai_action_arg=I +fai_reboot_arg=,reboot +case $1 in + -h|--help) + echo "see help from myfai-chboot" + exit 0 + ;; + -S) + fai_action_arg=S + fai_reboot_arg= + shift + ;; +esac + +[[ $EUID == 0 ]] || exec sudo "${BASH_SOURCE}" "$@" + +e() { + echo "$*" + if ! "$@"; then + echo "$0: error: exit code $? from: $*" + exit 1 + fi +} + +host=$1 + +type -t host &>/dev/null || apt-get -y install dnsutils +gateway_if=$(ip route | sed -rn 's/^default via \S+ dev (\S+) .*/\1/p') +if [[ ! $gateway_if ]]; then + echo "$0: failed to find gateway interface" + exit 1 +fi +# assuming ipv4, or else we might need to deal with multiple addresses +# in an ipv4 + ipv6 network. +network=$(ip -4 -o a show dev $gateway_if | sed -rn '/scope.*global/s/^(\S+\s+){3}(\S+)\s.*/\2/p') +if [[ ! $network ]]; then + echo "$0: failed to find network" + exit 1 +fi +my_ip=${network%/*} +if [[ $host == default ]]; then + ip=$network +elif [[ $host == [0-9]*.[0-9]*.[0-9]*.[0-9]* ]]; then + ip=$host +else + ip=$(host $host | sed -rn 's/^\S+ has address //p;T;q')/32 +fi + + +# alternate way of getting my ip +#gateway_ip=$(ip route | sed -rn 's/^default via (\S+) .*/\1/p') +#my_ip=$(host faiserver $gateway_ip | sed -rn 's/^\S+ has address //p;T;q') + +if modprobe nfsd &>/dev/null; then + std_arg="-u nfs://faiserver/srv/fai/config" + root_arg="$my_ip:/srv/fai/nfsroot" + # fai-setup without -e sets the ip to the local_ip/local_network, eg 192.168.1.3/24 + # I restrict it to one ip as simple but imperfect access control. + sed -ri --follow-symlinks '\%^/srv/fai/%d' /etc/exports + cat >>/etc/exports < + Deny from all + Allow from $ip + +EOF +fi + +rm -f /srv/tftp/fai/pxelinux.cfg/* +if [[ ! $1 ]]; then + exit 0 +fi + + +# man page doesn't explain this, but this deletes & thus disables +# all chboot systems. +e fai-chboot -${fai_action_arg}v $std_arg default # set it to default to get a val out of it next +kernel=$(fai-chboot -L '^default$' | awk '{print $3}') +default_k_args=$(fai-chboot -L '^default$' | \ + sed -r "s/^(\S+\s+){3}(.*)/\2/") +# example of default_k_args +# initrd=initrd.img-3.16.0-4-amd64 ip=dhcp root=192.168.1.3:/srv/fai/nfsroot aufs FAI_CONFIG_SRC=nfs://faiserver/srv/fai/config FAI_ACTION=install + +k_args=() +for arg in $default_k_args; do + case $arg in + # default root arg is /srv/fai/nfsroot + root=*) k_args+=(root=$root_arg) ;; + *) k_args+=($arg) ;; + esac +done +rm -f /srv/tftp/fai/pxelinux.cfg/* +e fai-chboot -k "${k_args[*]}" -v -f verbose,sshd,createvt$fai_reboot_arg $std_arg $kernel "$host" diff --git a/pxe-server b/pxe-server new file mode 100755 index 0000000..63373e3 --- /dev/null +++ b/pxe-server @@ -0,0 +1,185 @@ +#!/bin/bash +# Copyright (C) 2016 Ian Kelling + +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License +# as published by the Free Software Foundation; either version 2 +# of the License, or (at your option) any later version. + +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. + +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. + +# Setup dhcp server to point to tftp server, +# and depending on the type, setup the tftp server. + +# usage: $0 TYPE +# default distro is the base debian/fedora type. others are fai & arch. +# for no pxe server, use a no-op like : or true. + + +x="$(readlink -f "$BASH_SOURCE")"; source "${x%/*}/bash-trace" + + +usage() { + cat < tftpboot + + +Note: Uses GNU getopt options parsing style +EOF + exit $1 +} + +##### begin command line parsing ######## + +redep=true +acks=2 +wait=false +temp=$(getopt -l help harSw "$@") || usage 1 +eval set -- "$temp" +while true; do + case $1 in + -a) wait=true; set=false; shift ;; + -r) redep=false; shift ;; + -S) chboot_arg=-S; shift ;; + -w) wait=true; set=true; shift ;; + -h|--help) usage ;; + --) shift; break ;; + *) echo "$0: Internal error!" ; exit 1 ;; + esac +done + +read -r host type <<<"$@" + +case $# in + 0|2);; + *) + echo "$0: error: expected 0 or 2 arguments" + echo + usage 1 + ;; +esac + +if [[ $host && $host != default ]]; then + host_tag="tag:$host," +fi + +##### end command line parsing ######## + +e() { + echo "$@" + "$@" +} + +arch() { + cat </dev/null ||: + fi +fi + +if $wait; then + # fai's debian jessie 8.5ish does 2 dhcp requests when booting, + # roughly 4 seconds apart. Earlier + # versions did just 1. Now testing on a vm, it does 1. + # bleh. + echo "waiting for $acks dhcp acks then disabling pxe" + ack-wait $acks + type= + set-pxe + + # previously tried waiting for one more ack then disabling faiserver, + # since it can contain sensitive info, so turn it off when not in use, + # but disabling that for now as it's inconvenient to clean this + # up and run it in the background etc. + + # if [[ $type == fai ]]; then + # echo "waiting for 1 dhcp ack then disabling fai server" + # ack-wait 1 + # faiserver-disable + # fi +fi diff --git a/wrt-disabled-firewall-rules b/wrt-disabled-firewall-rules new file mode 100644 index 0000000..3d1e902 --- /dev/null +++ b/wrt-disabled-firewall-rules @@ -0,0 +1,145 @@ +firewall rules, temporarily disabled until I get them working + + +# each port forward needs corresponding forward in the vpn server + + +#http/https + + + +config redirect + option name bittorrent + option src vpn + option src_dport 63324 + option dest_ip 192.168.1.2 + option dest lan +# making the port open (not sure if this is actually needed) +config rule + option src vpn + option target ACCEPT + option dest_port 63324 + +config redirect + option name frodobittorrent + option src vpn + option src_dport 63326 + option dest_ip 192.168.1.3 + option dest lan +config rule + option src vpn + option target ACCEPT + option dest_port 63326 + + +config redirect + option name treetowlsyncthing + option src vpn + option src_dport 22000 + option dest_ip 192.168.1.2 + option dest lan + option proto tcp +config rule + option src vpn + option target ACCEPT + option dest_port 22000 + + +config redirect + option name bithtpc + option src vpn + option src_dport 63325 + option dest_ip 192.168.1.4 + option dest lan +config rule + option src vpn + option target ACCEPT + option dest_port 63325 + + + +#### begin rules for nfs #### +# https://serverfault.com/questions/377170/which-ports-do-i-need-to-open-in-the-firewall-to-use-nfs +# https://wiki.debian.org/SecuringNFS +# I had no /etc/default/quota, or any process named quota anything, +# so, assumed that was unneeded. seems to work. +config redirect + option src wan + option src_dport 111 + option dest_ip 192.168.1.2 + option dest lan +config rule + option src wan + option target ACCEPT + option dest_port 111 +config redirect + option src wan + option src_dport 2049 + option dest_ip 192.168.1.2 + option dest lan +config rule + option src wan + option target ACCEPT + option dest_port 2049 +config redirect + option src wan + option src_dport 32764 + option dest_ip 192.168.1.2 + option dest lan +config rule + option src wan + option target ACCEPT + option dest_port 32764 +config redirect + option src wan + option src_dport 32765 + option dest_ip 192.168.1.2 + option dest lan +config rule + option src wan + option target ACCEPT + option dest_port 32765 +config redirect + option src wan + option src_dport 32766 + option dest_ip 192.168.1.2 + option dest lan +config rule + option src wan + option target ACCEPT + option dest_port 32766 +config redirect + option src wan + option src_dport 32767 + option dest_ip 192.168.1.2 + option dest lan +config rule + option src wan + option target ACCEPT + option dest_port 32767 +config redirect + option src wan + option src_dport 32768 + option dest_ip 192.168.1.2 + option dest lan +config rule + option src wan + option target ACCEPT + option dest_port 32768 +#### end rules for nfs #### + + +config redirect + option name mariadb + option src wan + option src_dport 3306 + option dest lan + option dest_ip 192.168.1.2 + option proto tcp +config rule + option src wan + option target ACCEPT + option dest_port 3306 + option proto tcp + + diff --git a/wrt-setup b/wrt-setup new file mode 100755 index 0000000..d721f6a --- /dev/null +++ b/wrt-setup @@ -0,0 +1,383 @@ +#!/bin/bash +# Copyright (C) 2016 Ian Kelling + +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License +# as published by the Free Software Foundation; either version 2 +# of the License, or (at your option) any later version. + +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. + +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. + +x="$(readlink -f "$BASH_SOURCE")"; source "${x%/*}/bash-trace" + +pmirror() { + # background: upgrading all packages is not recommended because it + # doesn't go into the firmware. build new firmware if you want + # lots of upgrades. + f=(/tmp/opkg-lists/*) + if ! (( $(date -r $f +%s) + 60*60*24 > $(date +%s) )); then + opkg update + fi +} + +pi() { + for x in "$@"; do + if [[ ! $(opkg list-installed "$x") ]]; then + pmirror + opkg install "$@" + fi + done +} + +v() { + printf "+ %s\n" "$*" + "$@" +} + +cat >/usr/bin/arch-pxe-mount <<'EOFOUTER' +#!/bin/bash +# symlinks are collapsed for nfs mount points, so use a bind mount. +# tried putting this in /etc/config/fstab, +# then doig block mount, it didn't work. This doesn't persist across reboots, +# todo: figure that out +d=/run/archiso/bootmnt +cat > /etc/fstab </dev/null || mount $d +/etc/init.d/nfsd restart +EOFOUTER +chmod +x /usr/bin/arch-pxe-mount + +cat >.profile <<'EOF' +# changing login shell emits spam on ssh single commands & scp + # sed -i 's#/bin/ash$#/bin/bash#' /etc/passwd +#https://dev.openwrt.org/ticket/13852 +[ "$PS1" = "" ] || { + /bin/bash + exit +} +EOF +v pi kmod-usb-storage block-mount kmod-fs-ext4 nfs-kernel-server \ + tcpdump openvpn-openssl + + + +sed -ri "s/option[[:space:]]*encryption[[:space:]]*'?none'?/option encryption psk2\n option key pictionary49/" /etc/config/wireless +sed -i '/^[[:space:]]*option disabled/d' /etc/config/wireless +v wifi + + +v /etc/init.d/fstab enable ||: + +# rebooting makes mounting work, but comparing lsmod, +# i'm guessing this will too. todo, test it. +# 255 == module already loaded +for mod in scsi_mod sd_mod; do v modprobe $mod || [[ $? == 255 ]]; done + +# for arch pxe. The default settings in the installer expect to find +# the NFS at /run/archiso/bootmnt +mkdir -p /run/archiso/bootmnt + +# todo: at some later time, i found /mnt/usb not mounted, watch to see if +# that is the case after running this or rebooting. +# wiki says safe to do in case of fstab changes: +cedit /etc/config/fstab <<'EOF' || { v block umount; v block mount; } +config global automount + option from_fstab 1 + option anon_mount 1 + +config global autoswap + option from_fstab 1 + option anon_swap 1 + +config mount + option target /mnt/usb + option device /dev/sda2 + option fstype ext4 + option options rw,async,noatime,nodiratime + option enabled 1 + option enabled_fsck 0 + +config swap + option device /dev/sda1 + option enabled 1 + +EOF + + + +# exportfs -ra wont cut it when its the same path, but now a bind mount +cedit /etc/exports <<'EOF' || v /etc/init.d/nfsd restart ||: +/mnt/usb 192.168.1.0/255.255.255.0(rw,no_root_squash,insecure,sync,no_subtree_check) +# for arch pxe +/run/archiso/bootmnt 192.168.1.0/255.255.255.0(rw,no_root_squash,insecure,sync,no_subtree_check) +EOF + + +v /etc/init.d/portmap start +v /etc/init.d/nfsd start +v /etc/init.d/portmap enable +v /etc/init.d/nfsd enable + + + + + + +######### uci example:####### +# # https://wiki.openwrt.org/doc/uci +# wan_index=$(uci show firewall | sed -rn 's/firewall\.@zone\[([0-9])+\]\.name=wan/\1/p') +# wan="firewall.@zone[$wan_index]" +# if [[ $(uci get firewall.@forwarding[0].dest) != $forward_dest ]]; then +# # default is wan +# v uci set firewall.@forwarding[0].dest=$forward_dest +# uci commit firewall +# firewall_restart=true +# fi + + + +########## openvpn exampl +########## missing firewall settings for routing lan +########## traffic +# v /etc/init.d/openvpn start +# v /etc/init.d/openvpn enable + +# # from https://wiki.openwrt.org/doc/uci/firewall +# # todo: not sure if /etc/init.d/network needs restarting. +# # I did, and I had to restart the vpn afterwards. +# # This maps a uci interface to a real interface which is +# # managed outside of uci. +# v cedit /etc/config/network <<'EOF' ||: +# config interface 'tun0' +# option ifname 'tun0' +# option proto 'none' +# EOF +# v cedit /etc/config/openvpn <<'EOF' || v /etc/init.d/openvpn restart +# config openvpn my_client_config +# option enabled 1 +# option config /etc/openvpn/client.conf +# EOF + + +v cedit /etc/config/network <<'EOF' || v /etc/init.d/network reload +config 'route' 'transmission' + option 'interface' 'lan' + option 'target' '10.173.0.0' + option 'netmask' '255.255.0.0' + option 'gateway' '192.168.1.2' +EOF + +v cedit /etc/config/firewall <<'EOF' || firewall_restart=true +config redirect + option name ssh + option src wan + option src_dport 22 + option dest_ip 192.168.1.2 + option dest lan +config rule + option src wan + option target ACCEPT + option dest_port 22 + +config redirect + option name sshalt + option src wan + option src_dport 2222 + option dest_port 22 + option dest_ip 192.168.1.3 + option dest lan +config rule + option src wan + option target ACCEPT + option dest_port 2222 + +config redirect + option src wan + option src_dport 443 + option dest lan + option dest_ip 192.168.1.2 + option proto tcp +config rule + option src wan + option target ACCEPT + option dest_port 443 + option proto tcp + +config redirect + option src wan + option src_dport 1194 + option dest lan + option dest_ip 192.168.1.2 + option proto udp +config rule + option src wan + option target ACCEPT + option dest_port 1194 + option proto udp + + +config redirect + option src wan + option src_dport 80 + option dest lan + option dest_ip 192.168.1.2 + option proto tcp +config rule + option src wan + option target ACCEPT + option dest_port 80 + option proto tcp + +config redirect + option name syncthing + option src wan + option src_dport 22001 + option dest_ip 192.168.1.2 + option dest lan +config rule + option src wan + option target ACCEPT + option dest_port 22001 + + + +EOF + + + + +dnsmasq_restart=false +v cedit /etc/hosts < + +# default dhcp range is 100-150 +dhcp-host=f4:6d:04:02:ed:66,set:treetowl,192.168.1.2,treetowl +dhcp-host=00:26:18:97:bb:16,set:frodo,192.168.1.3,frodo +dhcp-host=10:78:d2:da:29:22,set:htpc,192.168.1.4,htpc +dhcp-host=00:1f:16:16:39:24,set:x2,192.168.1.5,x2 +# this is so fai can have an explicit name to use for testing, +# or else any random machine which did a pxe boot would get +# reformatted. The mac is from doing a virt-install, cancelling it, +# and copying the generated mac, so it should be randomish. +dhcp-host=52:54:00:9c:ef:ad,set:demohost,192.168.1.6,demohost +#dhcp-host=52:54:00:56:09:f9,set:faiserver,192.168.1.7,faiserver +dhcp-host=80:fa:5b:1c:6e:cf,set:tp,192.168.1.8,tp +dhcp-host=c4:43:8f:f2:79:1f,set:n5,192.168.1.9,n5 +# this is the ip it picks by default if dhcp fails, +# so might as well use it. +# hostname is the name it uses according to telnet +dhcp-host=b4:75:0e:94:29:ca,set:switch9429ca,192.168.1.251,switch9429ca + + +# template +# dhcp-host=,192.168.1., + +# Just leave the tftp server up even if we aren't doing pxe boot. +# It has no sensitive info. +enable-tftp=br-lan +tftp-root=/mnt/usb/tftpboot +EOF + +if $dnsmasq_restart; then + v /etc/init.d/dnsmasq restart +fi + +if $firewall_restart; then + v /etc/init.d/firewall restart +fi diff --git a/wrt-setup-remote b/wrt-setup-remote new file mode 100755 index 0000000..3dadb14 --- /dev/null +++ b/wrt-setup-remote @@ -0,0 +1,45 @@ +#!/bin/bash +# Copyright (C) 2016 Ian Kelling + +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License +# as published by the Free Software Foundation; either version 2 +# of the License, or (at your option) any later version. + +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. + +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. + + +set -eE -o pipefail +trap 'echo "$0:$LINENO:error: \"$BASH_COMMAND\" returned $?" >&2' ERR + +x="$(readlink -f "$BASH_SOURCE")"; cd ${x%/*} + +usage() { + cat <