From: Ian Kelling Date: Sun, 16 Apr 2017 00:44:58 +0000 (-0700) Subject: Move independent repo myunison into this repo X-Git-Url: https://iankelling.org/git/?a=commitdiff_plain;h=c12fecb2eb85890400d451732a2acecce5b69bc4;hp=9e52740ac533ee18f5907a648023dfcfa97c11c1;p=distro-setup Move independent repo myunison into this repo Merge remote-tracking branch 'myunison/master' --- diff --git a/LICENSE b/LICENSE new file mode 100644 index 0000000..d645695 --- /dev/null +++ b/LICENSE @@ -0,0 +1,202 @@ + + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright [yyyy] [name of copyright owner] + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. diff --git a/README b/README new file mode 100644 index 0000000..aef53eb --- /dev/null +++ b/README @@ -0,0 +1,33 @@ +~2.5k lines of bash to setup all my computers + +Initial os install is also automated using a separate repo called +automated-distro-installer, also at iankelling.org/git. + +This is not meant for other people to run verbatum, but for them to read +and copy the good parts. It has dependencies on other repos at +https://iankelling.org/git. + + +The main thing missing is any automation for the directory structure +those repos live in. So you would need to lookout for paths starting with +/a and adjust them. + +Background: reasoning behind using /a: The home directory is typically +used for local software development, but I use paths like /a instead, +for the following reasons: + +1. I want to run code directly from where I work on it, instead of +always having to create and use some install process. If that code is in +a home directory, and you want to run it as root (especially in contexts +where SUDO_USER is not set, like cron/systemd), you have to hardcode the +username for /home/username, or create some install process where that +username is saved somewhere, and then you are stuck with a single +username. If it ever got packaged for a gnu/linux distro, it would rely +on a hardcoded path with no username in it, so let's just do that. + +2. The home directory is inconvenient. It's filled with a bunch of junk +you don't care about, which makes directory listing horrible, makes it so +you can't back it up easily (for example, gvfs mountpoint in it breaks +lots of things), and has things you don't want to backup. So, you could +use a subdirectory. But typing /s is much faster than ~/s and in every +root context, /home/username/s. diff --git a/btrbk-run b/btrbk-run new file mode 100755 index 0000000..4642a70 --- /dev/null +++ b/btrbk-run @@ -0,0 +1,254 @@ +#!/bin/bash +# Copyright (C) 2016 Ian Kelling + +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at + +# http://www.apache.org/licenses/LICENSE-2.0 + +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +set -eE -o pipefail +trap 'echo "$0:$LINENO:error: \"$BASH_COMMAND\" returned $?" >&2' ERR + +[[ $EUID == 0 ]] || exec sudo -E "$BASH_SOURCE" "$@" +usage() { + echo "top of script file:" + sed -n '1,/^[# ]*end command line/{p;b};q' "$0" + exit $1 +} + +script_dir=$(dirname $(readlink -f "$BASH_SOURCE")) + +# note q is owned by root:1000 +# note p/m is owned 1000:1000 and chmod 700 +mountpoints=(/a) +private_mountpoints=(/q) +rsync_mountpoint=/q + +conf_only=false +dry_run=false # mostly for testing +resume_arg= +do_i=true +do_o=true + +temp=$(getopt -l help hcinoprt: "$@") || usage 1 +eval set -- "$temp" +while true; do + case $1 in + -c) conf_only=true; shift ;; + -i) do_i=false; shift ;; + -n) dry_run=true; dry_run_arg=-n; shift ;; + -o) do_o=false; shift ;; + -p) progress_arg="--progress"; shift ;; + # btrbk arg: Resume only. Skips snapshot creation. + -r) resume_arg=-r; shift ;; + -t) IFS=, targets=($2); unset IFS; shift 2 ;; + -h|--help) usage ;; + --) shift; break ;; + *) echo "$0: Internal error!" ; exit 1 ;; + esac +done + +if $do_o; then + private_mountpoints+=(/o) +fi +read primary <<<"$@" # not yet used + +##### end command line parsing ######## + +rsync-dirs() { + local host=$1 + local path=$2 + m rsync $dry_run_arg -ahi --relative --delete "$path" "root@$host:/" +} + +vol-conf() { + cat >>/etc/btrbk.conf <>/etc/btrbk.conf <>/etc/btrbk.conf </dev/null; then + echo "$0: error: no btrbk binary found" +fi + +cat >/etc/btrbk.conf <<'EOF' +ssh_identity /root/.ssh/id_rsa +# Just a guess that local7 is a good facility to pick. +# It's a bit odd that the transaction log has to be logged to +# a file or syslog, while other output is sent to std out. +# The man does not mention a way for them to be together, but +# I dunno if setting a log level like warn might also output +# transaction info. +transaction_syslog local7 + +# so we only run one at a time +lockfile /var/lock/btrbk.lock + +# default format of short does not accomidate hourly preservation setting +timestamp_format long-iso + +# only make a snapshot if things have changed +snapshot_create onchange +# I could make this different from target_preserve, +# if one disk had less space. +# for now, keeping them equal. +snapshot_preserve 36h 14d 8w 24m +snapshot_preserve_min 4h +snapshot_dir btrbk + +# so, total backups = ~89 +target_preserve 36h 14d 8w 24m +target_preserve_min 4h + +# if something fails and it's not obvious, try doing +# btrbk -l debug -v dryrun +EOF + +for mp in ${private_mountpoints[@]}; do # private mountpoints + if awk '{print $2}' /etc/fstab | grep -xF $mp &>/dev/null; then + mountpoints+=($mp) + fi +done + +# if our mountpoints are from stale snapshots, +# it doesn't make sense to do a backup. +check-subvol-stale ${mountpoints[@]} || exit 1 + +if [[ ! $targets ]]; then + case $HOSTNAME in + tp|x2) + if ! timeout -s 9 10 ssh frodo :; then + targets=($HOME_DOMAIN) + fi + ;; + treetowl) + targets=(frodo) + if timeout -s 9 10 ssh x2 :; then + targets+=(x2) + fi + ;; + *) + targets=(frodo) + ;; + esac + echo "targets: ${targets[*]}" +fi + + +# for i, we just do a 1 way sync from master to backup, +# and manually manage any changes to that. +i_possible=false +for tg in ${targets[@]}; do + # for an initial run, btrbk requires the dir to exist + ssh root@$tg mkdir -p /mnt/root/btrbk + if [[ $tg == frodo && $HOSTNAME == treetowl ]]; then + i_possible=true + fi +done +if ! $i_possible; then + do_i=false +fi + + +vol=/mnt/root +vol-conf +for m in ${mountpoints[@]}; do + sub=${m##*/} + sub-conf + for tg in ${targets[@]}; do + tg-conf + done +done + +if $do_i; then + vol=/mnt/iroot + vol-conf + sub=i + sub-conf + tg=frodo + vol=/mnt/root + tg-conf +fi + + + +# todo: umount first to ensure we don't have any errors +# todo: do some kill fuser stuff to make umount more reliable +# todo: run this on a systemd timer on $primary, once per hour, +# and if primary is, change that timer over to primary, and make +# sure we mount the latest + + + +if $conf_only; then + exit +fi + +if $dry_run; then + m btrbk -n $resume_arg run +else + # -q and just using the syslog option seemed nice, + # but it doesn't show when a send has a parent and when it doesn't. + m btrbk $progress_arg $resume_arg run +fi + +# if we have it, sync to systems which don't +if mountpoint $rsync_mountpoint >/dev/null; then + for tg in ${targets[@]}; do + case $tg in + tp|li|lk) + for x in /p/c/machine_specific/*.hosts; do + if grep -qxF $tg $x; then + dir=${x%.hosts} + rsync-dirs $tg $dir + fi + done + ;; + esac + done +fi + +if ! $dry_run; then + m $script_dir/mount-latest-remote ${targets[@]} +fi + + +# todo: move variable data we don't care about backing up +# to /nocow and symlink it. + + +# background on btrbk timezones. with short/long, timestamps use local time. +# for long, if your local time moves backwards, by moving timezones or +# for an hour when daylight savings changes it, you will temporarily get +# a more aggressive retention policy for the overlapping period, and +# vice versa for the opposite timezone move. The alternative is using +# long-iso, which puts timezone info into the timestamp, which means +# that instead of shifting time, you shift the start of day/week/month +# which is used for retention to your new local time, which means for +# example, if you moved forward by 8 hours, the daily/weekly/monthly +# retention will be 8 hours more aggressive since midnight is at a new +# time, unless you fake the timzeone using the TZ env variable. +# However, in the short term, there will be no inconsistencies. +# I don't see any problem with shifting when the day starts for +# retention, so I'm using long-iso. + +# note to create a long-iso timestamp: date +%Y%m%dT%H%M%S%z diff --git a/check-subvol-stale b/check-subvol-stale new file mode 100644 index 0000000..1277182 --- /dev/null +++ b/check-subvol-stale @@ -0,0 +1,87 @@ +#!/bin/bash +# Copyright (C) 2016 Ian Kelling +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# usage: $0 SUBVOL_MOUNTPOINT... +# +# In git, this is not not executable because it's meant to be installed +# using ./install-my-scripts +# +# If latest subvols $@ are not mounted, exit 1, print message, and touch +# /nocow/btrfs-stale/$subvol +# +# Either SUBVOL_MOUNTPOINT is a snapshot of the latest, or +# the latest snapshot is snapshot of SUBVOL_MOUNTPOINT. + +[[ $EUID == 0 ]] || exec sudo -E "$BASH_SOURCE" "$@" + +set -eE -o pipefail +trap 'echo "$0:$LINENO:error: \"$BASH_COMMAND\" returned $?" >&2' ERR + +shopt -s nullglob + +if [[ ! $@ ]]; then + echo "$0: error: expected mountpoint argument" +fi + +ret=0 +for d; do + vol=${d##*/} + cd /mnt/root/btrbk + snaps=($vol.20*) # Assumes we are in the 21st century. + if [[ ! $snaps ]]; then + # no snapshots yet + continue + fi + # when a btrbk bugfix makes it into the distro, + # we might replace this with btrbk list latest /mnt/root/$vol | ... + # note: this is duplicated in mount-latest-subvol + last_snap=$( + for f in ${snaps[@]}; do + printf "%s %s\n" $(date -d $(sed -r 's/(.{4})(..)(.{5})(..)(.*)/\1-\2-\3:\4:\5/' <<<${f#$vol.}) +%s) $f + done | sort -r | head -n 1 | awk '{print $2}' + ) + if [[ ! $last_snap ]]; then + # should not happen. + echo "$0: error: could not find latest snapshot for $d among ${snaps[@]}" + ret=1 + continue + fi + stale=true + # check that $d has $last_snap as a snapshot, + # or else $d is a snapshot of $last_snap. In the second + # case, we use a uuid comparison, which if I remember from the + # docs, is a bit more robust, perhaps to renames. + if btrfs sub show $d 2>/dev/null | sed '0,/^\s*Snapshot(s):/d;s/^\s*//' | \ + grep -xF btrbk/$last_snap &>/dev/null; then + stale=false + else + last_uuid=$(btrfs sub show $last_snap| awk '$1 == "UUID:" {print $2}') + if btrfs sub show $d| grep "^\s*Parent UUID:\s*$last_uuid$" &>/dev/null; then + stale=false + fi + fi + stale_dir=/nocow/btrfs-stale + stale_file=$stale_dir/$vol + if $stale; then + mkdir -p $stale_dir + printf "%s\n" $last_snap > $stale_file + echo "$d stale" + ret=1 + continue + else + rm -f $stale_file + fi +done +exit $ret diff --git a/conflink b/conflink new file mode 100755 index 0000000..8431c16 --- /dev/null +++ b/conflink @@ -0,0 +1,83 @@ +#!/bin/bash + +source /a/bin/errhandle/errcatch-function +source /a/bin/errhandle/bash-trace-function + +errcatch + +m() { + echo "$*" + "$@" +} +s() { sudo "$@"; } +lnf() { /a/exe/lnf "$@"; } + + +shopt -s nullglob +shopt -s extglob # note, already set with bash -l + +# If we make a link back to the root, we stop going deeper into subdir_files. +# This makes it so we can do subdir directories. +# +# Also note, under filesystem/, symlinks are expanded. + +subdir-link-r() { + local root="$1" + local targets=() + if [[ $2 ]]; then + targets=( "$2"/!(.git|..|.) ) + else + for f in "$1"/!(.git|..|.); do + [[ -d $f ]] && targets+=("$f") ||: + done + fi + local below="$( readlink -f "$root/..")" + for path in "${targets[@]}"; do + local fullpath="$(readlink -f "$path")" + #e $fullpath $below # debug + if [[ -f $path || $(dirname $(readlink -f "$fullpath")) == "$below" ]]; then + m lnf -T "$path" "$HOME/${path#$root/}" + elif [[ -d "$path" ]]; then + subdir-link-r "$root" "$path" + fi + done +} + +common-file-setup() { + local dir fs x + for dir in "$@"; do + fs=$dir/filesystem + if [[ -e $fs && $USER == ian ]]; then + # note, symlinks get resolved, not copied. + m s cp -RLT --preserve=mode,timestamps $fs / + fi + if [[ -e $dir/subdir_files ]]; then + subdir-link-r $dir/subdir_files + fi + local x=( $dir/!(subdir_files|filesystem|machine_specific|..|.) ) + (( ${#x[@]} >= 1 )) || continue + m lnf ${x[@]} ~ + done +} + +all_dirs=({/a/c,/p/c}{,/machine_specific/$HOSTNAME}) +# note, we assume a group of hosts does not have the +# same name as a single host, which is no problem on our scale. +for x in /p/c/machine_specific/*.hosts; do + if grep -qxF $HOSTNAME $x; then all_dirs+=( ${x%.hosts} ); fi +done + +c_dirs=(/a/c{,/machine_specific/$HOSTNAME}) +case $USER in + ian) + # p needs to go first so .ssh link is created, then config link inside it + common-file-setup ${all_dirs[@]} + sudo -u traci "$BASH_SOURCE" + ;; + traci) + common-file-setup ${c_dirs[@]} + ;; + *) + echo "$0: error: unexpected user"; exit 1 + ;; +esac diff --git a/desktop-20-autostart.sh b/desktop-20-autostart.sh new file mode 100755 index 0000000..a86864a --- /dev/null +++ b/desktop-20-autostart.sh @@ -0,0 +1,52 @@ +#!/bin/bash -l +# Copyright (C) 2016 Ian Kelling + +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at + +# http://www.apache.org/licenses/LICENSE-2.0 + +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# first 2 alternatives showed under ubuntu 14.04, second 2 under arch at 11/2015 + +if [[ $1 ]]; then + right_monitor_rotation=left +else + right_monitor_rotation=normal +fi + + +xout="$(xrandr)" +xe() { echo "$xout"; } +x=$(xe | grep -E '^(DisplayPort-[0123]|DVI-0|DP-[1234]|DVI-I-1) connected' | wc -l) +if (( x > 2 )); then + left=$(xe | sed -rn 's/^(DVI[^ ]+) connected .*/\1/p') + dps=( $(xe | sed -rn 's/^(DP-[01234]|DisplayPort-[01234]) connected .*/\1/p') ) + + middle=${dps[1]} + right=${dps[0]} + # on older distros, i needed to swap middle and right. + + + xrandr --output $left --mode 2560x1600 --pos 0x0 --rotate left \ + --output $middle --mode 2560x1600 --pos 1600x0 --rotate left \ + --output $right --mode 2560x1600 --pos 3200x0 --rotate $right_monitor_rotation +elif (( x == 2 )); then + # 3rd monitor not working atm, so doing this. + left=$(xe | sed -rn 's/^(DVI[^ ]+) connected .*/\1/p') + middle=$(xe | sed -rn 's/^(DP-[01234]|DisplayPort-[01234]) connected .*/\1/p') + xrandr --output $left --mode 2560x1600 --pos 0x0 --rotate left \ + --output $middle --mode 2560x1600 --pos 1600x0 --rotate left + +fi +/a/bin/distro-setup/input-setup +if isarch; then + pulseaudio --start +fi +date "+%A, %B %d, %r, %S seconds" > /tmp/desktop-20-autostart-log diff --git a/distro-begin b/distro-begin new file mode 100755 index 0000000..71e2a56 --- /dev/null +++ b/distro-begin @@ -0,0 +1,776 @@ +#!/bin/bash -l +# Copyright (C) 2016 Ian Kelling +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + + + +# for bootstrapping a new machine + +# in case we need it, +# to make ssh interactive shell run better, we run this first. +sudo bash -c 'source /a/c/repos/bash/.bashrc && source /a/exe/ssh-emacs-setup' + + +# usage: $0 [-r] HOSTNAME + +# tips: +# run any sudo command first so your pass is cached +# set the scrollback to unlimited in case something goes wrong + +if [[ $EUID == 0 ]]; then + if getent passwd ian; then + echo "$0: error: running as root. unprivileged user exists. use it." + exit 1 + else + echo "$0: warning: running as root. I will setup users then exit" + fi +fi + +interactive=true # set this to false to force set -x +[[ $- == *i* ]] || interactive=false + +if ! $interactive; then + set -x + set -e -o pipefail +fi +set -E +trap 'echo "$0:$LINENO:error: \"$BASH_COMMAND\" returned $?"' ERR + +exec &> >(sudo tee -a /var/log/distro-begin) +echo "$0: $(date): starting now)" + +# headless=false # unused atm +recompile=false +# for copying to a new data fs +bootstrapfs=false # old flag, needs new look before using. +while [[ $1 == -* ]]; do + case $1 in + -r) recompile=true; shift ;; + esac +done + +if [[ $1 ]]; then + export HOSTNAME=$1 +fi + +for f in iank-dev htpc treetowl x2 frodo tp li lj demohost; do + eval "$f() { [[ $HOSTNAME == $f ]]; }" +done +has_p() { treetowl || x2 || frodo || tp || demohost; } +has_x() { ! linode; } +linode() { lj || li; } +has_btrfs() { ! linode; } +home_network() { ! linode; } +encrypted() { has_p; } + +shopt -s extglob +export GLOBIGNORE=*/.:*/.. +umask 0002 + + +####### end command line parsing + +PATH="/a/exe:$PATH" +sed="sed --follow-symlinks" + +##### begin setup encryption scripts ###### +if encrypted; then + # I tried making a service which was dependent on reboot.target, + # but it happened too late in the shutdown process. + sudo dd of=/etc/systemd/system/keyscripton.service <<'EOF' +[Unit] +Description=Turn on automatic decryption of drives on boot +# tried using graphical.target, but it made my display manager restart before rebooting. +# generally, I don't think targets order shutdown like they do startup. +# So, I did systemd-analyze plot > something.svg, and picked a reliably started +# service that happens late in the game. +After=ntp.service +DefaultDependencies=no +# not sure if needed, makes sure we shut down before reboot.target +Conflicts=reboot.target + +[Service] +Type=oneshot +RemainAfterExit=yes +ExecStart=/bin/true +ExecStop=/a/exe/keyscript-on + +[Install] +WantedBy=keyscriptoff.service +EOF + sudo systemctl daemon-reload # needed if the file was already there + sudo systemctl stop keyscripton.service + # sudo systemctl start keyscripton.service + sudo systemctl enable keyscripton.service + + sudo dd of=/etc/systemd/system/keyscriptoff.service <<'EOF' +[Unit] +Description=Turn off automatic decryption of drives on boot + +[Service] +Type=oneshot +ExecStart=/a/exe/keyscript-off + +[Install] +WantedBy=multi-user.target +EOF + sudo systemctl daemon-reload # needed if the file was already there + sudo systemctl enable keyscriptoff.service + sudo systemctl start keyscriptoff.service +fi +##### end setup encryption scripts ###### + + +install-myqueue + +# this script has been designed to be idempotent +# todo, it would be nice to cut down on some of the output + + +for x in /a/bin/errhandle/*-function; do + source $x +done + + +set +e +$interactive || errcatch +set +x +source /a/bin/distro-functions/src/identify-distros +$interactive || set -x + +if isfedora; then + # comment out line disallowing calling sudo in scripts + sudo $sed -i 's/^Defaults *requiretty/#\0 # ian commented/' /etc/sudoers + # turn on magic sysrq commands for this boot cycle + echo 1 > sudo dd of=/proc/sys/kernel/sysrq + # selinux is not user friendly. Like, you enable samba, but you haven't run the magic selinux commands so it doesn't work + # and you have no idea why. + sudo $sed -i 's/^\(SELINUX=\).*/\1disabled/' /etc/selinux/config + selinuxenabled && sudo setenforce 0 +fi + + +# already ran for pxe installs, but used for vps & updates +distro=$(distro-name) +case $distro in + ubuntu|debian) + sudo bash -c ". /a/bin/fai/fai-wrapper && /a/bin/fai/fai/config/scripts/GRUB_PC/11-ian" + ;; + *) + sudo bash -c ". /a/bin/fai/fai-wrapper && +/a/bin/fai/fai/config/distro-install-common/end" + ;; +esac + +if linode; then + sudo $sed -i '/^127\.0\.1\.1/d' /etc/hosts + echo "127.0.1.1 $HOSTNAME.lan $HOSTNAME" | sudo tee -a /etc/hosts +fi + + +if [[ $EUID == 0 ]]; then + echo "$0: running as root. exiting now that users are setup" + exit 0 +fi + + +# this needs to be before installing pacserve so we have gpg conf. +conflink + +set +x +errallow +source ~/.bashrc +$interactive || errcatch +$interactive || set -x + + +# passwordless sudo +tu /etc/sudoers <<'EOF' +ian ALL=(ALL) NOPASSWD: ALL +Defaults env_keep += SUDOD +EOF + + +# enable magic sysrq keys. debian docs say it is already enabled by default +isfedora && tu /etc/sysctl.conf 'kernel.sysrq = 1' + + +if isdebian; then + codename=$(debian-codename) + if isdebian-stable && has_x; then + s dd of=/etc/apt/sources.list.d/mozilla-iceweasel.list <$x + sudo dd of=/etc/pacman.conf if=$x; rm $x + sudo systemctl enable pacserve.service + sudo systemctl start pacserve.service + + # strange error if just installing trash-cli: "pyalpm requires python", + # so I see that it requires python2, and installing that manually fixes it. + # I didn't see this on earlier installation, main thing which changed was + # pacserve, so not sure if it's related. + pi python2 +fi + +pup +pi trash-cli + + +###### link files ########### +# convenient to just do all file linking in one place + +# if it wasn't set already, we could set hostname here +#echo treetowl | s dd of=/etc/hostname +#s hostname -F /etc/hostname +#HOSTNAME=$(hostname) + + +s lnf -T /a/bin /b + +if has_p; then + lnf -T /p/News ~/News +fi + +s lnf /q/root/.editor-backups /q/root/.undo-tree-history \ + /a/opt /a/c/.emacs.d $HOME/mw_vars /k/backup /root + +rootsshsync + +s lnf /a/c/.inputrc /a/c/.vim /a/c/.vimrc /a/c/.gvimrc /root + +# machine is going away +# if [[ $HOSTNAME == htpc ]]; then +# lnf -T /i/Videos ~/Downloads +# fi + +if has_p; then + # for dovecot + lnf -T /i/k/mboxes ~/mail +fi + + +# basic needed packages +case $(distro-name) in + debian) + if has_x; then + if isdebian-stable; then + pi firefox/$codename-backports + else + # for a while, firefox/unstable did not have + # dependencies satisfied by testing packages, and i hit + # a conflict, it wanted a newer libfontconfig1, but + # emacs build-deps wanted an older one. In this case, + # I switch to using firefox-esr. note: They seem + # to release a new esr version every 9 months or so. + pi firefox/unstable + s dd of=/etc/apt/preferences.d/firefox <<'EOF' +Package: firefox +Pin: release a=unstable +Pin-Priority: 500 +EOF + fi + fi + # for hosts which require nonfree drivers + # i previously had extra packages listed here linux-image-amd64 + # firmware-linux-free linux-headers-amd64, but I + # don\'t see any reason why. seems to work in testing without. + # remove this note if it continues to work. + p=firmware-linux-nonfree + if apt-cache show $p &>/dev/null; then + pi $p + fi + ;;& + ubuntu|debian) + if has_x; then + if isdebian-stable; then + pi xmacro + else + pi xmacro/unstable # has no unstable deps + fi + pi gtk-redshift xinput + fi + ;;& + fedora) + p -y groupinstall development-tools c-development books admin-tools + pi wget man-pages + if has_x; then + pi redshift-gtk + # debian has this package patched to work, upstream is dead + # tried using alien, pi alien, alien -r *.deb, rpm -Uhv *.rpm, got this error, so fuck it + # file /usr/bin from install of xmacro-0.3pre_20000911-7.x86_64 conflicts with file from package filesystem-3.2-19.fc20.x86_64 + # http://packages.debian.org/source/sid/xmacro + pi patch libXtst-devel + cd $(mktemp -d) + wget http://ftp.de.debian.org/debian/pool/main/x/xmacro/xmacro_0.3pre-20000911.orig.tar.gz + wget http://ftp.de.debian.org/debian/pool/main/x/xmacro/xmacro_0.3pre-20000911-6.diff.gz + ex *.gz + patch -p0 < xmacro_0.3pre-20000911-6.diff + cd xmacro-0.3pre-20000911.orig + make + sleep 1 # not sure why the following command couldn\'t find, so trying this + # no make install target + s cp -f xmacroplay xmacrorec xmacrorec2 /usr/local/bin + fi + ;;& + arch) + # like apt-cache + pi pkgfile + s pkgfile --update + if has_x; then + # libxtst is missing dep https://aur.archlinux.org/packages/xmacro/#news + pi xorg-server redshift xorg-xinput libxtst xmacro + + # background: + # https://aur.archlinux.org/packages/xkbset/#comment-545419 + cert=$(mktemp) + cat >$cert <<'EOF' +-----BEGIN CERTIFICATE----- +MIIJADCCB+igAwIBAgIRAIVAhZ0TMbQ5jTm0koI8X6YwDQYJKoZIhvcNAQELBQAw +djELMAkGA1UEBhMCVVMxCzAJBgNVBAgTAk1JMRIwEAYDVQQHEwlBbm4gQXJib3Ix +EjAQBgNVBAoTCUludGVybmV0MjERMA8GA1UECxMISW5Db21tb24xHzAdBgNVBAMT +FkluQ29tbW9uIFJTQSBTZXJ2ZXIgQ0EwHhcNMTUxMjA4MDAwMDAwWhcNMTgxMjA3 +MjM1OTU5WjCBsTELMAkGA1UEBhMCVVMxDjAMBgNVBBETBTY1MjExMREwDwYDVQQI +EwhNaXNzb3VyaTERMA8GA1UEBxMIQ29sdW1iaWExHzAdBgNVBAkTFjExMDAgQ2Fy +cmllIEZyYW5ja2UgRHIxHzAdBgNVBAoTFlVuaXZlcnNpdHkgb2YgTWlzc291cmkx +CzAJBgNVBAsTAk1VMR0wGwYDVQQDExRmYWN1bHR5Lm1pc3NvdXJpLmVkdTCCASIw +DQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAN8Kap8hASpxQeqjHibGsCR1PBkh +nW9p5FkuhGpMW/3ko8QfxH0W1Hq2y2DTFUmq17kH3GfT3h9a7HcmUrC3q15PciOB +WR3j8u0bDfVppyAZXiHJzYGN7xHiPrZtFEGgwZd28+sW80WXTbGl+zKkmeZguGdH +AVGeWJEFK44ctLbpjHWCy+xNuhxJuL4olwPoV7WX9IUhceC0rxYQANhLGOJhbchj +Z76MA8dc2K3CZI5m7VqQwl09QSnCfz00afUr88ny9vj1S5k2ADS46gaE9O0lM6EY +z/uZvMizXN/4ko+hFBjCSt0Vhxjx0kYDSP15btiwh700ywBEubpvLROmd48CAwEA +AaOCBUswggVHMB8GA1UdIwQYMBaAFB4Fo3ePbJbiW4dLprSGrHEADOc4MB0GA1Ud +DgQWBBTTNWrSb+V/Ayy0i8W2LExMUisQMzAOBgNVHQ8BAf8EBAMCBaAwDAYDVR0T +AQH/BAIwADAdBgNVHSUEFjAUBggrBgEFBQcDAQYIKwYBBQUHAwIwZwYDVR0gBGAw +XjBSBgwrBgEEAa4jAQQDAQEwQjBABggrBgEFBQcCARY0aHR0cHM6Ly93d3cuaW5j +b21tb24ub3JnL2NlcnQvcmVwb3NpdG9yeS9jcHNfc3NsLnBkZjAIBgZngQwBAgIw +RAYDVR0fBD0wOzA5oDegNYYzaHR0cDovL2NybC5pbmNvbW1vbi1yc2Eub3JnL0lu +Q29tbW9uUlNBU2VydmVyQ0EuY3JsMHUGCCsGAQUFBwEBBGkwZzA+BggrBgEFBQcw +AoYyaHR0cDovL2NydC51c2VydHJ1c3QuY29tL0luQ29tbW9uUlNBU2VydmVyQ0Ff +Mi5jcnQwJQYIKwYBBQUHMAGGGWh0dHA6Ly9vY3NwLnVzZXJ0cnVzdC5jb20wggOg +BgNVHREEggOXMIIDk4IUZmFjdWx0eS5taXNzb3VyaS5lZHWCGmFkdmlzaW5nLmNv +YXMubWlzc291cmkuZWR1ghBhaGEubWlzc291cmkuZWR1ghZhbGwtY3JhZnQubWlz +c291cmkuZWR1gh1hbWVyaWNhbmJhc2tldHJ5Lm1pc3NvdXJpLmVkdYIXYW5kcmVh +cmlldy5taXNzb3VyaS5lZHWCFWFydGdyYWRzLm1pc3NvdXJpLmVkdYIYYmFja3Vw +LmNvYXMubWlzc291cmkuZWR1ghBiaWMubWlzc291cmkuZWR1ghZibG9nLmNvYXMu +bWlzc291cmkuZWR1ghVjb3dhbmxhYi5taXNzb3VyaS5lZHWCFWRhZS5zdGF0Lm1p +c3NvdXJpLmVkdYIRZGljZS5taXNzb3VyaS5lZHWCIGRpZ2l0YWxzdG9yeXRlbGxp +bmcubWlzc291cmkuZWR1gg9lYS5taXNzb3VyaS5lZHWCG2Vib29rLWRldi5tYXRo +Lm1pc3NvdXJpLmVkdYIXZWJvb2suZWNvbi5taXNzb3VyaS5lZHWCGGVuZ2xpc2g4 +MDA2Lm1pc3NvdXJpLmVkdYIZZXVnZW5lZml0c2NoLm1pc3NvdXJpLmVkdYIYZXVy +b2t1bHR1cmUubWlzc291cmkuZWR1ghNmY2RsYWIubWlzc291cmkuZWR1ghZnZW9t +dXNldW0ubWlzc291cmkuZWR1ghRoYXJzdGFkLm1pc3NvdXJpLmVkdYITbHVkd2ln +Lm1pc3NvdXJpLmVkdYIYbWFjaGluZXNob3AubWlzc291cmkuZWR1ghNtYWpvcnMu +bWlzc291cmkuZWR1ghBtZ2EubWlzc291cmkuZWR1ghdvcmdhbnByaW50Lm1pc3Nv +dXJpLmVkdYIUcGh5c2ljcy5taXNzb3VyaS5lZHWCFHBtLmNoZW0ubWlzc291cmku +ZWR1ghxyZWNydWl0aW5nLmVjb24ubWlzc291cmkuZWR1ghdyZXBlYy5lY29uLm1p +c3NvdXJpLmVkdYIUc2NhbmxhYi5taXNzb3VyaS5lZHWCFnNzc2MuY29hcy5taXNz +b3VyaS5lZHWCF3RlYWNoLmNvYXMubWlzc291cmkuZWR1ghd0b3B0ZWFjaGVyLm1p +c3NvdXJpLmVkdYIQdnNmLm1pc3NvdXJpLmVkdYIid2hpdGVwYXBlci5ncmFkc2No +b29sLm1pc3NvdXJpLmVkdTANBgkqhkiG9w0BAQsFAAOCAQEAQutYVAqG7MpmG2Nu +Z/UypjYkN4JvwRbKBpTrce2IT/Sy29x6chBbyD+0WE6QORBtaUHuzE1KoXqpnF4M +QrkKw0oBAC6x9dISoomq0DkIndtoBYYLaxSoII6F4OGWgF7pQ/7MiCBYzsKQpn9t +aofMcTfvnCjq+MCIaeYnUKBVww0lOJlUxZGKxFJvRpf78HfbBauojjRO2zXLZD/u +KMspbTfDaj5etIgWGShY2eml3N/SjAENmZYkcgDBYFyi8CckcEBAVzpH1+D+7Anz +txHSYDNHAYLv83MwbegApa1FwPqlG/4SdEU8G6e6Xf5GLC/6GPGVTUpr7o348OOO +lzGQzw== +-----END CERTIFICATE----- +-----BEGIN CERTIFICATE----- +MIIF+TCCA+GgAwIBAgIQRyDQ+oVGGn4XoWQCkYRjdDANBgkqhkiG9w0BAQwFADCB +iDELMAkGA1UEBhMCVVMxEzARBgNVBAgTCk5ldyBKZXJzZXkxFDASBgNVBAcTC0pl +cnNleSBDaXR5MR4wHAYDVQQKExVUaGUgVVNFUlRSVVNUIE5ldHdvcmsxLjAsBgNV +BAMTJVVTRVJUcnVzdCBSU0EgQ2VydGlmaWNhdGlvbiBBdXRob3JpdHkwHhcNMTQx +MDA2MDAwMDAwWhcNMjQxMDA1MjM1OTU5WjB2MQswCQYDVQQGEwJVUzELMAkGA1UE +CBMCTUkxEjAQBgNVBAcTCUFubiBBcmJvcjESMBAGA1UEChMJSW50ZXJuZXQyMREw +DwYDVQQLEwhJbkNvbW1vbjEfMB0GA1UEAxMWSW5Db21tb24gUlNBIFNlcnZlciBD +QTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAJwb8bsvf2MYFVFRVA+e +xU5NEFj6MJsXKZDmMwysE1N8VJG06thum4ltuzM+j9INpun5uukNDBqeso7JcC7v +HgV9lestjaKpTbOc5/MZNrun8XzmCB5hJ0R6lvSoNNviQsil2zfVtefkQnI/tBPP +iwckRR6MkYNGuQmm/BijBgLsNI0yZpUn6uGX6Ns1oytW61fo8BBZ321wDGZq0GTl +qKOYMa0dYtX6kuOaQ80tNfvZnjNbRX3EhigsZhLI2w8ZMA0/6fDqSl5AB8f2IHpT +eIFken5FahZv9JNYyWL7KSd9oX8hzudPR9aKVuDjZvjs3YncJowZaDuNi+L7RyML +fzcCAwEAAaOCAW4wggFqMB8GA1UdIwQYMBaAFFN5v1qqK0rPVIDh2JvAnfKyA2bL +MB0GA1UdDgQWBBQeBaN3j2yW4luHS6a0hqxxAAznODAOBgNVHQ8BAf8EBAMCAYYw +EgYDVR0TAQH/BAgwBgEB/wIBADAdBgNVHSUEFjAUBggrBgEFBQcDAQYIKwYBBQUH +AwIwGwYDVR0gBBQwEjAGBgRVHSAAMAgGBmeBDAECAjBQBgNVHR8ESTBHMEWgQ6BB +hj9odHRwOi8vY3JsLnVzZXJ0cnVzdC5jb20vVVNFUlRydXN0UlNBQ2VydGlmaWNh +dGlvbkF1dGhvcml0eS5jcmwwdgYIKwYBBQUHAQEEajBoMD8GCCsGAQUFBzAChjNo +dHRwOi8vY3J0LnVzZXJ0cnVzdC5jb20vVVNFUlRydXN0UlNBQWRkVHJ1c3RDQS5j +cnQwJQYIKwYBBQUHMAGGGWh0dHA6Ly9vY3NwLnVzZXJ0cnVzdC5jb20wDQYJKoZI +hvcNAQEMBQADggIBAC0RBjjW29dYaK+qOGcXjeIT16MUJNkGE+vrkS/fT2ctyNMU +11ZlUp5uH5gIjppIG8GLWZqjV5vbhvhZQPwZsHURKsISNrqOcooGTie3jVgU0W+0 ++Wj8mN2knCVANt69F2YrA394gbGAdJ5fOrQmL2pIhDY0jqco74fzYefbZ/VS29fR +5jBxu4uj1P+5ZImem4Gbj1e4ZEzVBhmO55GFfBjRidj26h1oFBHZ7heDH1Bjzw72 +hipu47Gkyfr2NEx3KoCGMLCj3Btx7ASn5Ji8FoU+hCazwOU1VX55mKPU1I2250Lo +RCASN18JyfsD5PVldJbtyrmz9gn/TKbRXTr80U2q5JhyvjhLf4lOJo/UzL5WCXED +Smyj4jWG3R7Z8TED9xNNCxGBMXnMete+3PvzdhssvbORDwBZByogQ9xL2LUZFI/i +eoQp0UM/L8zfP527vWjEzuDN5xwxMnhi+vCToh7J159o5ah29mP+aJnvujbXEnGa +nrNxHzu+AGOePV8hwrGGG7hOIcPDQwkuYwzN/xT29iLp/cqf9ZhEtkGcQcIImH3b +oJ8ifsCnSbu0GB9L06Yqh7lcyvKDTEADslIaeSEINxhO2Y1fmcYFX/Fqrrp1WnhH +OjplXuXE0OPa0utaKC25Aplgom88L2Z8mEWcyfoB7zKOfD759AN7JKZWCYwk +-----END CERTIFICATE----- +-----BEGIN CERTIFICATE----- +MIIF3jCCA8agAwIBAgIQAf1tMPyjylGoG7xkDjUDLTANBgkqhkiG9w0BAQwFADCB +iDELMAkGA1UEBhMCVVMxEzARBgNVBAgTCk5ldyBKZXJzZXkxFDASBgNVBAcTC0pl +cnNleSBDaXR5MR4wHAYDVQQKExVUaGUgVVNFUlRSVVNUIE5ldHdvcmsxLjAsBgNV +BAMTJVVTRVJUcnVzdCBSU0EgQ2VydGlmaWNhdGlvbiBBdXRob3JpdHkwHhcNMTAw +MjAxMDAwMDAwWhcNMzgwMTE4MjM1OTU5WjCBiDELMAkGA1UEBhMCVVMxEzARBgNV +BAgTCk5ldyBKZXJzZXkxFDASBgNVBAcTC0plcnNleSBDaXR5MR4wHAYDVQQKExVU +aGUgVVNFUlRSVVNUIE5ldHdvcmsxLjAsBgNVBAMTJVVTRVJUcnVzdCBSU0EgQ2Vy +dGlmaWNhdGlvbiBBdXRob3JpdHkwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIK +AoICAQCAEmUXNg7D2wiz0KxXDXbtzSfTTK1Qg2HiqiBNCS1kCdzOiZ/MPans9s/B +3PHTsdZ7NygRK0faOca8Ohm0X6a9fZ2jY0K2dvKpOyuR+OJv0OwWIJAJPuLodMkY +tJHUYmTbf6MG8YgYapAiPLz+E/CHFHv25B+O1ORRxhFnRghRy4YUVD+8M/5+bJz/ +Fp0YvVGONaanZshyZ9shZrHUm3gDwFA66Mzw3LyeTP6vBZY1H1dat//O+T23LLb2 +VN3I5xI6Ta5MirdcmrS3ID3KfyI0rn47aGYBROcBTkZTmzNg95S+UzeQc0PzMsNT +79uq/nROacdrjGCT3sTHDN/hMq7MkztReJVni+49Vv4M0GkPGw/zJSZrM233bkf6 +c0Plfg6lZrEpfDKEY1WJxA3Bk1QwGROs0303p+tdOmw1XNtB1xLaqUkL39iAigmT +Yo61Zs8liM2EuLE/pDkP2QKe6xJMlXzzawWpXhaDzLhn4ugTncxbgtNMs+1b/97l +c6wjOy0AvzVVdAlJ2ElYGn+SNuZRkg7zJn0cTRe8yexDJtC/QV9AqURE9JnnV4ee +UB9XVKg+/XRjL7FQZQnmWEIuQxpMtPAlR1n6BB6T1CZGSlCBst6+eLf8ZxXhyVeE +Hg9j1uliutZfVS7qXMYoCAQlObgOK6nyTJccBz8NUvXt7y+CDwIDAQABo0IwQDAd +BgNVHQ4EFgQUU3m/WqorSs9UgOHYm8Cd8rIDZsswDgYDVR0PAQH/BAQDAgEGMA8G +A1UdEwEB/wQFMAMBAf8wDQYJKoZIhvcNAQEMBQADggIBAFzUfA3P9wF9QZllDHPF +Up/L+M+ZBn8b2kMVn54CVVeWFPFSPCeHlCjtHzoBN6J2/FNQwISbxmtOuowhT6KO +VWKR82kV2LyI48SqC/3vqOlLVSoGIG1VeCkZ7l8wXEskEVX/JJpuXior7gtNn3/3 +ATiUFJVDBwn7YKnuHKsSjKCaXqeYalltiz8I+8jRRa8YFWSQEg9zKC7F4iRO/Fjs +8PRF/iKz6y+O0tlFYQXBl2+odnKPi4w2r78NBc5xjeambx9spnFixdjQg3IM8WcR +iQycE0xyNN+81XHfqnHd4blsjDwSXWXavVcStkNr/+XeTWYRUc+ZruwXtuhxkYze +Sf7dNXGiFSeUHM9h4ya7b6NnJSFd5t0dCy5oGzuCr+yDZ4XUmFF0sbmZgIn/f3gZ +XHlKYC6SQK5MNyosycdiyA5d9zZbyuAlJQG03RoHnHcAP9Dc1ew91Pq7P8yF1m9/ +qS3fuQL39ZeatTXaw2ewh0qpKJ4jjv9cJ2vhsE/zB+4ALtRZh8tSQZXq9EfX7mRB +VXyNWQKV3WKdwrnuWih0hKWbt5DHDAff9Yk2dDLWKMGwsAvgnEzDHNb842m1R0aB +L6KCq9NjRHDEjf8tM7qtj3u1cIiuPhnPQCjY/MiQu12ZIvVS5ljFH4gxQ+6IHdfG +jjxDah2nGN59PRbxYvnKkKj9 +-----END CERTIFICATE----- +EOF + cat /etc/ssl/certs/ca-certificates.crt >> $cert + CURL_CA_BUNDLE=$cert pi xkbset + fi + + ;;& + ubuntu|debian|fedora) + if has_x; then + if isdebian-stable; then + pi xkbset + else + # xkbset was in testing for quite a while, dunno + # why it\'s not anymore. Sometime I should check and + # see if it\'s back in testing, but the unstable package + # doesn\'t upgrade anything form testing, and it\'s tiny + # so I\'m not bothering to automate it. + pi xkbset/unstable + fi + fi + ;;& +esac + +if has_x; then + pi xbindkeys +fi +pi cryptsetup lvm2 +# enables trim for volume delete, other rare commands. +sudo $sed -ri 's/( *issue_discards\b).*/\1 = 1/' /etc/lvm/lvm.conf + +if encrypted; then + if isdeb; then + sudo cp /usr/share/doc/util-linux/examples/fstrim.{service,timer} /etc/systemd/system + fi + # does weekly trim + sudo systemctl enable fstrim.timer +fi + +dirs=(/mnt/{1,2,3,4,5,6,7,8,9}) +s mkdir -p "${dirs[@]}" +s chown ian:ian "${dirs[@]}" + + +tu /etc/fstab <<'EOF' +/i/w /w none bind,noauto 0 0 +/i/k /k none bind,noauto 0 0 +EOF + + +if ! mountpoint /kr; then + s mkdir -p /kr + s chown ian:traci /kr +fi + +if home_network; then + if [[ $HOSTNAME == treetowl ]]; then + tu /etc/fstab <<'EOF' +/k /kr none bind,noauto 0 0 +EOF + else + tu /etc/fstab <<'EOF' +treetowl:/k /kr nfs noauto 0 0 +EOF + fi +fi + +s mkdir -p /q /i/{w,k} +for dir in /{i,w,k}; do + if mountpoint $dir; then continue; fi # already mounted + s mkdir -p $dir + s chown ian:ian $dir +done + +# not needed for all hosts, but rather just keep it uniform +s mkdir -p /mnt/iroot + +# debian auto mounting of multi-disk encrypted btrfs is busted. It is +# in jessie, and in stretch as of 11/26/2016 I have 4 disks in cryptab, +# based on 3 of those, it creates .device units for /dev/mapper/dev... +# then waits endlessly for them on bootup, after the /dev/mapper disks +# have already been created and exist. todo: create a simple repro +# for this in a vm and report it upstream. +if has_btrfs || home_network; then + pi nfs-common + s dd of=/root/imount <<'EOF' +#!/bin/bash +[[ $EUID == 0 ]] || exec sudo -E "$BASH_SOURCE" "$@" +set -eE -o pipefail +trap 'echo "$0:$LINENO:error: \"$BASH_COMMAND\" returned $?" >&2' ERR +for dir in /i /mnt/iroot /k /kr /w; do + if ! mountpoint $dir &>/dev/null && \ + awk '{print $2}' /etc/fstab | grep -xF $dir &>/dev/null; then + if awk '{print $3}' /etc/fstab | grep -xF nfs &>/dev/null; then + mount $dir || echo "warning: failed to mount nfs on $dir" + else + mount $dir + fi + fi +done +EOF + s chmod +x /root/imount + + s dd of=/etc/systemd/system/imount.service <<'EOF' +[Unit] +Description=Mount /i and related mountpoints + +[Service] +Type=oneshot +ExecStart=/root/imount + +[Install] +# note /kr needs networking, this target is the simplest way to +# time it when the network should be up, but not do something +# dumb like delay startup until the network is up. It happens +# at some time after network.target +WantedBy=multi-user.target +EOF + sudo systemctl daemon-reload # needed if the file was already there + sudo systemctl enable imount.service + sudo systemctl start imount.service +fi + +dir=/nocow +if has_btrfs; then + if ! mountpoint $dir; then + subvol=/mnt/root/nocow + if [[ ! -e $subvol ]]; then + s btrfs subvolume create $subvol + s chown root:1000 $subvol + s chattr +C $subvol + fi + + first_root_crypt=$(awk '$2 == "/" {print $1}' /etc/mtab) + tu /etc/fstab < >(sudo tee -a /var/log/distro-end) +echo "$0: $(date): starting now)" + +src="${BASH_SOURCE%/*}" + +end_msg() { + = local y + IFS= read -r -d '' y ||: + end_msg_var+="$y" +} + +spa() { # simple package add + simple_packages+=($@) +} + +distro=$(distro-name) + +pending_reboot=false +sed="sed --follow-symlinks" + +# template +case $distro in +esac + +pup + +simple_packages=( + htop + mailutils + nmon + rdiff-backup + ruby + ruby-rest-client + tree + vim + wcd +) + +case $HOSTNAME in + lj|li) : ;; + *) + # universal packages + # swh-plugins is for karaoke pulsaudio filter. + # mutagen for pithos + simple_packages+=( + apache2 + apache2-doc + apt-doc + aptitude-doc-en + bash-doc + binutils-doc + bwm-ng + chromium + cpio-doc + cloc + cron + debconf-doc + duplicity + eclipse + evince + fdupes + feh + filelight + gawk-doc + gcc-doc + gdb + gdb-doc + git-doc + git-email + gitk + glibc-doc + goaccess + gnome-screenshot + i3lock + iproute2-doc + jq + linux-doc + locate + make-doc + manpages + manpages-dev + meld + mumble + nmap + offlineimap + p7zip + paprefs + parted-doc + pavucontrol + pdfgrep + perl-doc + pianobar + pidgin + python3-doc + python3-mutagen + reportbug + sqlite3-doc + squashfs-tools + swh-plugins + tar-doc + tcpdump + transmission-remote-gtk + vlc + whois + ) + spa $(apt-cache search ruby[.0-9]+-doc| awk '{print $1}') + ;; +esac + + +########### begin section including li ################ + + +case $distro in + debian) + if [[ `debian-archive` == testing ]]; then + pi acme-tiny + fi +esac + +case $distro in + fedora) spa unrar ;; + *) spa unrar-free ;; +esac + + +case $distro in + arch) + # ubuntu 14.04 uses b-cron, + # but its not maintained in arch. + # of the ones in the main repos, cronie is only one maintained. + # fcron appears abandoned software. + pi cronie + sgo cronie + ;; + *) : ;; # other distros come with cron. +esac + + +case $distro in + debian|ubuntu) + pi debian-goodies + ;; +esac + + +case $distro in + *) pi at ;;& + arch) sgo atd ;; +esac + + +case $distro in + debian) pi curl;; + arch) : ;; + # fedora: unknown +esac + +case $distro in + # tk for gitk + arch) spa git tk ;; + *) spa git ;; +esac + +case $distro in + arch) spa the_silver_searcher ;; + debian|ubuntu) spa silversearcher-ag ;; + # fedora unknown +esac + +case $distro in + debian|ubuntu) spa ntp;; + arch) + pi ntp + sgo ntpd + ;; + # others unknown +esac + + +# no equivalent in other distros: +case $distro in + debian|ubuntu) + pi aptitude + if ! dpkg -s apt-file &>/dev/null; then + # this condition is just a speed optimization + pi apt-file + s apt-file update + fi + # for debconf-get-selections + spa debconf-utils + ;; +esac + +case $distro in + ubuntu|debian) spa ack-grep ;; + arch|fedora) spa ack ;; + # fedora unknown +esac + +case $distro in + arch|debian|ubuntu) + spa bash-completion + ;; + # others unknown +esac + + + + + +# disable motd junk. +case $(distro-name) in + debian) + # allows me to pipe with ssh -t, and gets rid of spam + # http://forums.debian.net/viewtopic.php?f=5&t=85822 + # i'd rather disable the service than comment the init file + # this says disabling the service, it will still get restarted + # but this script doesn't do anything on restart, so it should be fine + s dd of=/var/run/motd.dynamic if=/dev/null + # stretch doesn't have initscripts pkg installed by default + if [[ $(debian-codename) == jessie ]]; then + s update-rc.d motd disable + fi + ;; + ubuntu) + # this isn't a complete solution. It still shows me when updates are available, + # but it's no big deal. + s t /etc/update-motd.d/10-help-text /etc/update-motd.d/00-header + ;; +esac + +# automatic updates +# reference: +# https://debian-handbook.info/browse/stable/sect.regular-upgrades.html +# /etc/cron.daily/apt calls unattended-upgrades +# /usr/share/doc/unattended-upgrades# cat README.md +# /etc/apt/apt.conf.d/50unattended-upgrades +if isdebian; then + setup-debian-auto-update +fi + +# we've got a few dependencies later on, so install them now. +pi "${simple_packages[@]}" +simple_packages=() + +# website setup +case $HOSTNAME in + lj|li) + + case $HOSTNAME in + lj) domain=iank.bid; exit 0 ;; + li) domain=iankelling.org ;; + esac + /a/h/setup.sh $domain + /a/h/build.rb + + sudo -E /a/bin/mediawiki-setup/mw-setup-script + #$src/phab-setup + + pi-nostart mumble-server + s $sed -ri "s/^ *(serverpassword=).*/\1$(< /a/bin/bash_unpublished/mumble_pass)/" /etc/mumble-server.ini + sgo mumble-server + + vpn-server-setup -d + + sudo dd of=/etc/systemd/system/vpnmail.service <>/etc/hosts" | s tee -a $f + sgo openvpn + domain=cal.iankelling.org + acme-tiny-wrapper $domain + apache-site -f 10.8.0.4:5232 - $domain <<'EOF' +#https://httpd.apache.org/docs/2.4/mod/mod_authn_core.html#authtype + + Options +FollowSymLinks +Multiviews +Indexes + AllowOverride None + AuthType basic + AuthName "Authentication Required" + # setup one time, with root:www-data, 640 + AuthUserFile "/etc/caldav-htpasswd" + Require valid-user + +EOF + # nginx version of above would be: + # auth_basic "Not currently available"; + # auth_basic_user_file /etc/nginx/caldav/htpasswd; + + + ########## begin pump.io setup ########## + + # once pump adds a logrotation script, turn off nologger, + # and add + # "logfile": "/var/log/pumpio/pumpio.log", + # + s dd of=/etc/pump.io.json <<'EOF' +{ + "secret": "SECRET_REPLACE_ME", + "driver": "mongodb", + "params": { "dbname": "pumpio" }, + "noweb": false, + "site": "pump.iankelling.org", + "owner": "Ian Kelling", + "ownerURL": "https://iankelling.org/", + "port": 8001, + "urlPort": 443, + "hostname": "pump.iankelling.org", + "nologger": true, + "datadir": "/home/pumpio/pumpdata", + "enableUploads": true, + "debugClient": false, + "disableRegistration": true, + "noCDN": true, + "key": "/home/pumpio/pump.iankelling.org-domain.key", + "cert": "/home/pumpio/pump.iankelling.org-chained.pem", + "address": "localhost", + "sockjs": false +} +EOF + s sed -i "s#SECRET_REPLACE_ME#$(cat /p/c/machine_specific/li/pump-secret)#" /etc/pump.io.json + + # jessie\'s node is too old + # https://nodejs.org/en/download/package-manager/ + curl -sL https://deb.nodesource.com/setup_6.x | sudo -E bash - + pi nodejs + cd /home/ian + rm -rf pump.io.git + git clone https://github.com/pump-io/pump.io.git + cd pump.io + # note: doing this or the npm install pump.io as root had problems. + npm install + npm run build + # normally, next command would be + # s npm install -g databank-mongodb + # but it\'s this until a bug in pump gets fixed + s npm install -g databank-mongodb@0.19.2 + s useradd -m -s /bin/false pumpio + sudo -u pumpio mkdir -p /home/pumpio/pumpdata + # for testing browser when only listening to localhost, + # in the pump.io.json, set hostname localhost, urlPort 5233 + #ssh -L 5233:localhost:5233 li + acme-tiny-wrapper -c /home/pumpio pump.iankelling.org + + s mkdir -p /var/log/pumpio/ + s chown pumpio:pumpio /var/log/pumpio/ + + apache-site -c /home/pumpio - pump.iankelling.org <<'EOF' +# currently a bug in pump that we cant terminate ssl + SSLProxyEngine On + ProxyPreserveHost On + ProxyPass / https://127.0.0.1:8001/ + ProxyPassReverse / https://127.0.0.1:8001/ + # i have sockjs disabled per people suggesting that + # it won\'t work with apache right now. + # not sure if it would work with this, + # but afaik, this is pointless atm. + + ProxyPass wss://127.0.0.1:8001/main/realtime/sockjs/ + ProxyPassReverse wss://127.0.0.1:8001/main/realtime/sockjs/ + +EOF + + s dd of=/etc/systemd/system/pump.service <<'EOF' +[Unit] +Description=pump.io +After=syslog.target network.target + +[Service] +Type=simple +User=pumpio +Group=pumpio +ExecStart=/home/ian/pump.io/bin/pump +Environment=NODE_ENV=production +# failed to find databank-mongodb without this. +# I just looked at my environment variables took a guess. +Environment=NODE_PATH=/usr/lib/nodejs:/usr/lib/node_modules:/usr/share/javascript + +[Install] +WantedBy=multi-user.target +EOF + ser daemon-reload + sgo pump + ########## end pump.io setup ############ + + + ############# begin setup mastodon ############## + + # https://store.docker.com/editions/community/docker-ce-server-debian?tab=description + pi software-properties-common + curl -fsSL https://download.docker.com/linux/debian/gpg | sudo apt-key add - + sudo add-apt-repository \ + "deb [arch=amd64] https://download.docker.com/linux/debian \ + $(lsb_release -cs) \ + stable" + p update + pi docker-ce + sgo docker + # this may not be needed + ser start docker + + curl -L https://github.com/docker/compose/releases/download/1.12.0/docker-compose-`uname -s`-`uname -m` | s dd of=/usr/local/bin/docker-compose + s chmod +x /usr/local/bin/docker-compose + + # i subscrubed to https://github.com/docker/compose/releases.atom + # to deal with updates manually. + + cd ~ + i clone https://github.com/tootsuite/mastodon + cd mastodon + # https://github.com/tootsuite/mastodon/tree/v1.1.2 + # subbed to atom feed to deal with updates + i co v1.1.2 + + # per instructions, uncomment redis/postgres persistence in docker-compose.yml + sed -i 's/^#//' docker-compose.yml + + cat >.env.production <<'EOF' +REDIS_HOST=redis +REDIS_PORT=6379 +DB_HOST=db +DB_USER=postgres +DB_NAME=postgres +DB_PASS= +DB_PORT=5432 + +LOCAL_DOMAIN=mast.iankelling.org +LOCAL_HTTPS=true + +SINGLE_USER_MODE=true + +SMTP_SERVER=10.8.0.4 +SMTP_PORT=25 +SMTP_LOGIN=li +SMTP_FROM_ADDRESS=notifications@mast.iankelling.org +SMTP_DOMAIN=mast.iankelling.org +SMTP_DELIVERY_METHOD=smtp +EOF + + for key in PAPERCLIP_SECRET SECRET_KEY_BASE OTP_SECRET; do + printf "%s=%s" $key "$(docker-compose run --rm web rake secret)" >>.env.production + done + s cat /etc/mailpass| while read -r domain port pass; do + if [[ $domain == mail.iankelling.org ]]; then + printf "SMTP_PASSWORD=%s" "$pass" >>.env.production + break + fi + done + + + + docker-compose run --rm web rails assets:precompile + + # docker daemon takes care of starting on boot. + docker-compose up -d + + acme-tiny-wrapper mast.iankelling.org + s a2enmod proxy_wstunnel headers + apache-site -f 3000 - mast.iankelling.org <<'EOF' + ProxyPreserveHost On + RequestHeader set X-Forwarded-Proto "https" + ProxyPass /500.html ! + ProxyPass /oops.png ! + ProxyPass /api/v1/streaming/ ws://localhost:4000/ + ProxyPassReverse /api/v1/streaming/ ws://localhost:4000/ + ErrorDocument 500 /500.html + ErrorDocument 501 /500.html + ErrorDocument 502 /500.html + ErrorDocument 503 /500.html + ErrorDocument 504 /500.html +EOF + + + ############### !!!!!!!!!!!!!!!!! + ############### manual steps: + + # only following 2 people atm, so not bothering to figure out backups + # when mastodon has not documented it at all. + # + # fsf@status.fsf.org + # cwebber@toot.cat + # dbd@status.fsf.org + # johns@status.fsf.org + + # sign in page is at https://mast.iankelling.org/auth/sign_in + # register as iank, then + # https://github.com/tootsuite/documentation/blob/master/Running-Mastodon/Administration-guide.md + # docker-compose run --rm web bundle exec rails mastodon:make_admin USERNAME=iank + + ############# end setup mastodon ############## + + echo "$0: $(date): ending now)" + exit 0 + ;; +esac + + +########### end section including li/lj ############### + +if [[ $HOSTNAME == treetowl ]]; then + # note, see bashrc for more documentation. + pi rss2email + s dd of=/etc/systemd/system/rss2email.service <<'EOF' +[Unit] +Description=rss2email +After=multi-user.target + +[Service] +User=ian +Type=oneshot +# about 24 hours of failures +ExecStart=/a/bin/log-quiet/sysd-mail-once -288 rss2email r2e run +EOF + s dd of=/etc/systemd/system/rss2email.timer <<'EOF' +[Unit] +Description=rss2email + +[Timer] +# for initial run. required. +OnActiveSec=30 +# for subsequent runs. +OnUnitInactiveSec=300 + +[Install] +WantedBy=timers.target +EOF + s systemctl daemon-reload + sgo rss2email.timer +fi + +######### begin pump.io periodic backup ############# +if [[ $HOSTNAME == treetowl ]]; then + s dd of=/etc/systemd/system/pumpbackup.service <<'EOF' +[Unit] +Description=pump li backup +After=multi-user.target + +[Service] +User=ian +Type=oneshot +ExecStart=/a/bin/log-quiet/sysd-mail-once pump-backup /a/bin/distro-setup/pump-backup +EOF + s dd of=/etc/systemd/system/pumpbackup.timer <<'EOF' +[Unit] +Description=pump li backup hourly + +[Timer] +OnCalendar=hourly + +[Install] +WantedBy=timers.target +EOF + s systemctl daemon-reload + sgo pumpbackup.timer +fi +######### end pump.io periodic backup ############# + +case $distro in + debian|ubuntu) + # suggests because we want the resolvconf package. + # todo: check other distros to make sure it\'s installed + pi-nostart --install-suggests openvpn + # pi-nostart does not disable + ser disable openvpn + ;; + *) pi openvpn;; +esac + +if private-host; then + vpn-mk-client-cert -n mail li + cn=$(s openssl x509 -noout -nameopt multiline -subject \ + -in /etc/openvpn/client/mail.crt | \ + sed -rn 's/^\s*commonName\s*=\s*(.*)/\1/p') + echo "ifconfig-push 10.8.0.4 255.255.255.0" | \ + ssh root@li dd of=/etc/openvpn/client-config/"$cn" +fi +ser enable mailroute +if [[ $HOSTNAME == treetowl ]]; then + # note, this will need to be changed when the mail/contacts host changes + sgo openvpn-client@mail + /a/bin/distro-setup/radicale-setup +fi + +## android studio setup +# this contains the setting for android sdk to point to +# /a/opt/androidsdk, which is asked upon first run +lnf /a/opt/.AndroidStudio2.2 ~ +# android site says it needs a bunch of packages for ubuntu, +# but I googled for debian, and someone says you just need lib32stdc++6 plus the +# jdk +# https://pid7007blog.blogspot.com/2015/07/installing-android-studio-in-debian-8.html +# see w.org for more android studio details +spa lib32stdc++6 default-jdk + + +if [[ $HOSTNAME == treetowl ]]; then + ############# begin syncthing setup ########### + + # It\'s simpler to just worry about running it in one place for now. + # I assume it would work to clone it\'s config to another non-phone + # and just run it in one place instead of the normal having a + # separate config. I lean toward using the same config, since btrfs + # syncs between comps. + case $distro in + arch) pi syncthing ;; + ubuntu|debian) + # testing has relatively up to date packages + if ! isdebian-testing; then + # based on error when doing apt-get update: + # E: The method driver /usr/lib/apt/methods/https could not be found. + pi apt-transport-https + # google led me here: + # https://apt.syncthing.net/ + curl -s https://syncthing.net/release-key.txt | sudo apt-key add - + s="deb http://apt.syncthing.net/ syncthing release" + if [[ $(cat /etc/apt/sources.list.d/syncthing.list) != $s ]]; then + echo "$s" | s dd of=/etc/apt/sources.list.d/syncthing.list + p update + fi + fi + pi syncthing + ;; + esac + lnf -T /w/syncthing /home/ian/.config/syncthing + sgo syncthing@ian # runs as ian + + # these things persist in ~/.config/syncthing, which I save in + # /w/syncthing (not in /p, because syncthing should continue to + # run on home server even when using laptop as primary device) + # open http://localhost:8384/ + # change listen address from default to tcp://:22001, + # this is because we do port forward so it doesn\'t have to use + # some external server, but the syncthing is broken for port forward, + # you get a message, something "like connected to myself, this should not happen" + # when connecting to other local devices, so I bump the port up by 1, + # based on + # https://forum.syncthing.net/t/connected-to-myself-should-not-happen/1763/19. + # Without this, it was being stuck syncing at 0%. + # Set gui username and password. + # + # install syncthing via f-droid, + # folder setting, turn off master folder (makes it read only). + # on phone, add device, click bar code icon + # on dekstop, top right, actions, device id + # after adding, notification will appear on desktop to confirm + # + # syncing folder. from phone to desktop: select desktop in the + # folder on phone\'s sync options, notification will appear in + # desktop\'s web ui within a minute. For the reverse, the + # notification will appear in android\'s notifications, you have to + # swipe down and tap it to add the folder. It won\'t appear in the + # syncthing ui, which would be intuitive, but don\'t wait for it + # there. + # + # On phone, set settings to run syncthing all the time, and + # show no notification. + # + # Folder versioning would make sense if I didn\'t already use btrfs + # for backups. I would choose staggered, or trash can for more space. + # + # if needed to install on a remote comp: + # ssh -L 8384:localhost:8384 -N frodo + # open http://localhost:8384/ + # + # Note, the other thing i did was port forward port 22000, + # per https://docs.syncthing.net/users/firewall.html + + ############# end syncthing setup ########### +fi + + + +# no equivalent in other distros: +case $distro in + debian|ubuntu) + # for gui bug reporting + spa python-vte + ;; +esac + + +####### misc packages ########### + +if [[ $HOSTNAME == treetowl ]]; then + case $distro in + debian|ubuntu) + # note i had to do this, which is persistent: + # cd /i/k + # s chgrp debian-transmission torrents partial-torrents + + # syslog says things like + # 'Failed to set receive buffer: requested 4194304, got 425984' + # google suggets giving it even more than that + tu /etc/sysctl.conf<<'EOF' +net.core.rmem_max = 67108864 +net.core.wmem_max = 16777216 +EOF + s sysctl -p + + # some reason it doesn\'t seem to start automatically anyways + pi-nostart transmission-daemon + + # the folder was moved here after an install around 02/2017. + # it contains runtime data, + # plus a simple symlink to the config file which it\'s + # not worth separating out. + s lnf -T /i/transmission-daemon /var/lib/transmission-daemon/.config/transmission-daemon + # + # config file documented here, and it\'s the same config + # for daemon vs client, so it\'s documented in the gui. + # https://trac.transmissionbt.com/wiki/EditConfigFiles#Options + # + # I originaly setup rpc-whitelist, but after using + # routing to a network namespace, it doesn\'t see the + # real source address, so it\'s disabled. + # + # Changed the cache-size to 256 mb, reduces disk use. + # It is a read & write cache. + # + s ruby <<'EOF' +require 'json' +p = '/etc/transmission-daemon/settings.json' +File.write(p, JSON.pretty_generate(JSON.parse(File.read(p)).merge({ +'rpc-whitelist-enabled' => false, +'rpc-authentication-required' => false, +'incomplete-dir' => '/i/k/partial-torrents', +'incomplete-dir-enabled' => true, +'download-dir' => '/i/k/torrents', +"speed-limit-up" => 800, +"speed-limit-up-enabled" => true, +"peer-port" => 61486, +"cache-size-mb" => 256, +"ratio-limit" => 5.0, +"ratio-limit-enabled" => true, +})) + "\n") +EOF + + # make sure its not enabled, not sure if this is needed + ser disable transmission-daemon + sgo transmission-daemon-nn + ;; + # todo: others unknown + esac +fi + +# adapted from /var/lib/dpkg/info/transmission-daemon.postinst +if ! getent passwd debian-transmission > /dev/null; then + case $distro in + arch) + s useradd \ + --system \ + --create-home \ + --home-dir /var/lib/transmission-daemon \ + --shell /bin/false \ + debian-transmission + ;; + *) + s adduser --quiet \ + --system \ + --group \ + --no-create-home \ + --disabled-password \ + --home /var/lib/transmission-daemon \ + debian-transmission + ;; + esac +fi + +# dunno why it\'s there, but get rid of it +case $HOSTNAME in + li|lj) s rm -rf /home/linode ;; +esac + +# arch had a default config, +# debian had nothing until you start it. +# With a little trial an error, here is a minimal config +# taken from the generated one, plus changes that the +# settings ui does, without a bunch of ui crap settings. +# +# only settings I set were +# hostname +# auto-connect +# password + + +# the password is randomly generated on first run +rpc_pass=$(s ruby <<'EOF' +require 'json' +p = '/etc/transmission-daemon/settings.json' +puts JSON.parse(File.read(p))["rpc-password"] +EOF + ) + +for f in /home/*; do + d=$f/.config/transmission-remote-gtk + u=${f##*/} + s -u $u mkdir -p $d + s -u $u dd of=$d/config.json </dev/null; then + s apt-get -fy install + else + exit 1 + fi + ;; + esac + ;; + arch) + pi google-chrome + ;; + esac + ;; +esac + +# printer +case $distro in + arch) + pi cups ghostscript gsfonts # from arch wiki cups page + pi hplip # from google + s gpasswd -a $USER sys # from arch wiki + sgo org.cups.cupsd.service + # goto http://127.0.0.1:631 + # administration tab, add new printer button. + # In debian, I could use hte recommended driver, + # in arch, I had to pick out the 6L driver. + ;; + debian|ubuntu) + spa hplip + ;; + # other distros unknown +esac + + +case $distro in + ubuntu|debian) pi --no-install-recommends mairix notmuch ;; + fedora|arch) spa mairix notmuch ;; +esac +case $distro in + arch) spa nfs-utils ;; + ubuntu|debian) spa nfs-client ;; +esac +case $distro in + ubuntu|debian) spa par2 ;; + arch|fedora) spa par2cmdline ;; +esac + +# needed for my tex resume +case $distro in + ubuntu|debian) spa texlive-full ;; + arch) spa texlive-most ;; + # fedora unknown +esac + +case $distro in + ubuntu) + # flash, unrar, codecs, ms fonts. + # This has a manual prompt. + spa ubuntu-restricted-extras + ;; + fedora) + pi yum-utils + # rpm fusion recommended codecs + s su -c "yum localinstall -y --nogpgcheck http://download1.rpmfusion.org/free/fedora/rpmfusion-free-release-$(rpm -E %fedora).noarch.rpm http://download1.rpmfusion.org/nonfree/fedora/rpmfusion-nonfree-release-$(rpm -E %fedora).noarch.rpm" + pi gstreamer-plugins-ugly gstreamer-plugins-bad gstreamer-ffmpeg\ + xine-lib-extras-freeworld + ;; +esac + +case $distro in + # optional dep for firefox for h.264 video + arch) spa gst-libav ;; + # other distros, probably come by default +esac + +case $distro in + fedora|ubuntu|debian) spa gnupg-agent ;; + arch) : ;; +esac + + +case $distro in + fedora) spa pinentry-gtk ;; + *) : ;; # comes default or with other packages +esac + +case $distro in + arch) spa firefox pulseaudio;; + *) : ;; # comes default or with other packages +esac + + +case $distro in + arch) spa ttf-dejavu;; + debian|ubuntu) spa fonts-dejavu ;; + # others unknown +esac + + +case $distro in + arch) spa xorg-xev;; + debian|ubuntu) spa x11-utils ;; + # others unknown +esac + +case $distro in + arch) pi virt-install;;& + debian|ubuntu) pi virtinst ;;& + *) pi virt-manager ;; # creates the libvirt group in debian at least + # others unknown +esac +# allow user to run vms, from debian handbook +for x in ian traci; do s usermod -a -G libvirt,kvm $x; done +# bridge networking as user fails. google lead here, but it doesn\'t work: +# oh well, I give up. +# http://wiki.qemu.org/Features-Done/HelperNetworking +# s mkdir /etc/qemu +# f=/etc/qemu/bridge.conf +# s dd of=$f <<'EOF' +# allow br0 +# EOF +# #s chown root:qemu $f # debian has somethig like qemu-libvirt. equivalent? +# s chmod 640 $f + + +case $distro in + arch) spa cdrkit;; + debian|ubuntu) spa genisoimage;; + # others unknown +esac + +case $distro in + arch) spa spice-gtk3 ;; + debian|ubuntu) spa spice-client-gtk;; + # others unknown +esac + +# general known for debian/ubuntu, not for fedora + +case $distro in + debian|ubuntu) + pi golang-go + # a bit of googling, and added settings to bashrc + go get -u github.com/mvdan/fdroidcl/cmd/fdroidcl + ;; + # others unknown +esac + + +case $distro in + arch) + # cdrkit for cloud-init isos + # dnsmasq & ebtables for nat networking in libvirt + # qemu for qemu-img, bind-tools for dig + # dmidecode just because syslog complains + pi unzip xorg-xmodmap dmidecode ebtables\ + bridge-utils dnsmasq qemu bind-tools + # otherwise we get error about accessing kvm module. + # seems like there might be a better way, but google was a bit vague. + s $sed -ri '/^ *user *=/d' /etc/libvirt/qemu.conf + echo 'user = "root"' | s tee -a /etc/libvirt/qemu.conf + # https://bbs.archlinux.org/viewtopic.php?id=206206 + # # this should prolly go in the wiki + sgo virtlogd.socket + # guessing this is not needed + #sgo virtlogd.service + sgo libvirtd + + ;; +esac + +case $distro in + arch) pi virtviewer ;; + *) : ;; # other distros have it as a dependency afaik. +esac + + + +case $distro in + fedora) cabal install shellcheck ;; + *) spa shellcheck ;; + # unknown for older ubuntu +esac + + +case $distro in + arch|debian|ubuntu) spa pumpa ;; + # others unknown. do have a buildscript: + # /a/bin/buildscripts/pumpa ;; +esac + + +case $distro in + debian|ubuntu) spa android-tools-adbd/unstable ;; + arch) spa android-tools ;; + # other distros unknown +esac + +if [[ $HOSTNAME == treetowl ]]; then + case $distro in + debian) + if [[ `debian-archive` == testing ]]; then + # has no unstable dependencies + pi bitcoind/unstable + src=/a/opt/bitcoin/contrib/init/bitcoind.service + s cp $src /etc/systemd/system + p=/etc/bitcoin/bitcoin + dst=/etc/systemd/system/bitcoinjm.service + # jm for joinmarket + $sed -r "/^\s*ExecStart/s,${p}.conf,${p}jm.conf," $src \ + >/etc/systemd/system/bitcoinjm.service + + d=jm; jm=d # being clever for succinctness + for s in d jm; do + s $sed -ri "/^\s*\[Unit\]/a Conflicts=bitcoin${!s}.service" \ + /etc/systemd/system/bitcoin${s}.service + done + + ser daemon-reload + + dir=/nocow/.bitcoin + s mkdir -p $dir + s chown -R bitcoin:bitcoin $dir + dir=/etc/bitcoin + s mkdir -p $dir + s chown -R root:bitcoin $dir + s chmod 750 $dir + + # pruning decreases the bitcoin dir to 2 gb, keeps + # just the recent blocks. can\'t do a few things like + # import a wallet dump. + # pruning works, but people had to do + # some manual stuff in joinmarket. I dun need the + # disk space, so not bothering yet, maybe in a year or so. + # https://github.com/JoinMarket-Org/joinmarket/issues/431 + #https://bitcoin.org/en/release/v0.12.0#wallet-pruning + #prune=550 + + f=$dir/bitcoin.conf + s dd of=$f </dev/null < x.html +EOF + + +case $distro in + debian|ubuntu) + case `debian-archive` in + stable) + s dd of=/etc/apt/preferences.d/unison-gtk <<'EOF' +Explanation: Allow unison-gtk to be upgraded +Package: unison-gtk +Pin: release a=testing +Pin-Priority: 500 +EOF + # dont think using testing is needed since I figured out how to + # deal with mismatching unison compilers, but I dont + # see any reason to revert it, since it only installs + # a single package which is primarily a single binary + ;; + esac + pi unison/testing + pi unison-gtk/testing # after to make it the default unison + ;; + arch) + pi unison gtk2 + ;; +esac + +case $distro in + arch) + # default is alsa, doesn\'t work with with pianobar + s dd of=/etc/libao.conf <<'EOF' +default_driver=pulse +EOF + ;; +esac + +# note, for jessie, it depends on a higher version of btrfs-tools. +# +# # disabled due to my patch being in btrbk +# case $distro in +# arch|debian|ubuntu) pi btrbk ;; +# # others unknown +# esac +cd /a/opt/btrbk +s make install +spa pv # for progress bar when running interactively. +if [[ $HOSTNAME == treetowl ]]; then + # backup/sync manually on others hosts for now. + sgo btrbk.timer + # note: to see when it was last run, + # ser list-timers +fi + +if [[ $HOSTNAME == treetowl ]] && [[ `debian-archive` != testing ]]; then + # fail2 ban is broken, with a workaround, per + # https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=770171 + # ill wait a while to see if it gets fixed + pi fail2ban + sgo fail2ban +fi + + + + + +case $distro in + debian|ubuntu) s gpasswd -a ian adm ;; #needed for reading logs +esac + +# tor +case $distro in + # based on + # https://www.torproject.org/docs/rpms.html.en + # https://www.torproject.org/docs/debian.html.en + # todo: figure out if the running service needs to be restarted upon updates + + + # todo on fedora: setup non-dev packages + fedora) + s dd of=/etc/yum.repos.d/torproject.repo <<'EOF' +[tor] +name=Tor experimental repo +enabled=1 +baseurl=http://deb.torproject.org/torproject.org/rpm/tor-testing/fc/20/$basearch/ +gpgcheck=1 +gpgkey=http://deb.torproject.org/torproject.org/rpm/RPM-GPG-KEY-torproject.org.asc + +[tor-source] +name=Tor experimental source repo +enabled=1 +autorefresh=0 +baseurl=http://deb.torproject.org/torproject.org/rpm/tor-testing/fc/20/SRPMS +gpgcheck=1 +gpgkey=http://deb.torproject.org/torproject.org/rpm/RPM-GPG-KEY-torproject.org.asc +EOF + + # to be secure, take a look at the fingerprint reported from the following install, and see if it matches from the link above: + # 3B9E EEB9 7B1E 827B CF0A 0D96 8AF5 653C 5AC0 01F1 + sgo tor + /a/bin/buildscripts/tor-browser + ;; + ubuntu) + tu /etc/apt/sources.list "deb http://deb.torproject.org/torproject.org $(debian-codename) main" + gpg --keyserver keys.gnupg.net --recv 886DDD89 + gpg --export A3C4F0F979CAA22CDBA8F512EE8CBC9E886DDD89 | sudo apt-key add - + p update + pi deb.torproject.org-keyring + pi tor + /a/bin/buildscripts/tor-browser + ;; + debian) + pi tor + /a/bin/buildscripts/tor-browser + ;; + arch) + pi tor tor-browser-en + sgo tor + ;; + # ubuntu unknown +esac + +# nfs server +case $distro in + fedora) + end_msg <<'EOF' +fedora todo: disable the firewall or find a way to automate it. +there's an unused section in t.org for tramikssion firewall setup + +fedora manual config for nfs: +s firewall-config +change to permanent configuration +check the box for nfs +was hard to figure this out, not sure if this is all needed, but +unblock these too +mountd: udp/tcp 20048 +portmapper, in firewall-config its called rpc-bind: udp/tcp 111 +troubleshooting, unblock things in rpcinfo -p +make sure to reload the firewall to load the persistent configuration + + +EOF + pi nfs-utils + sgo nfs-server + ;; + debian|ubuntu) + pi nfs-server + ;; + arch) + pi nfs-utils || pending_reboot=true + sgo rpcbind + # this failed until I rebooted + sgo nfs-server + ;; +esac + +if [[ $HOSTNAME == treetowl ]]; then + # nohide = export filesystems mounted deeper than the export point + # fsid=0 makes this export the "root" export + # not documented in the man page, but this means + # 1. it can be mounted with a shorthand of server:/ + # 2. exports that are subdirectories of this one will automatically be mounted + tu /etc/exports <<'EOF' +/k 192.168.1.0/24(rw,fsid=0,nohide,no_root_squash,async,no_subtree_check,insecure) +EOF + s exportfs -rav +fi + + +e "$end_msg_var" + + +# persistent virtual machines + +case $distro in + debian|ubuntu) + pi libosinfo-bin; + ;; +esac + +# distro may not know about win 10 yet. +variant=win7 +if ! virt-install --os-variant list &>/dev/null; then # we are using a newer virt-install + for v in 10 8.1 8; do + if osinfo-query os | gr "^\s*win${v/./\\.}\s" &>/dev/null; then + variant=win$v + break + fi + done +fi + +if ! s virsh list --all --name | grep -xF win10 &>/dev/null; then + + # created account with + # win10vmian@outlook.com, and easy to remember password + # win 10 virtio, makes disk way way way faster + # wget https://fedorapeople.org/groups/virt/virtio-win/direct-downloads/latest-virtio/virtio-win.iso + # https://wiki.archlinux.org/index.php/QEMU#Change_Existing_Windows_VM_to_use_virtio + # for installing virtio after initial install instead of with initial iso: + # qemu-img create -f qcow2 fake.qcow2 1G + # --disk=/a/images/virtio-win.iso,device=cdrom \ + # --disk=/a/images/fake.qcow2,bus=virtio + # Also, + # went to device manager, saw 2 pci devices with yellow !, + # did search for drivers, pick cdrom location, done. + # + # from http://www.tenforums.com/tutorials/4189-fast-startup-turn-off-windows-10-a.html. + # google said there was a control panel option for it, but + # that turned out to be a lie. + # Put this in a .bat file and run as administrator to turn off + # hyberboot which fucks things up. + # REG ADD "HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\Power" /V HiberbootEnabled /T REG_dWORD /D 0 /F + # power settings, turn off display: never + # run "control userpasswords2", turn on automatic login. + # note: when changing devices, I just undefine, the create the vm again. + + if [[ -e /nocow/user/vms/win10.qcow2 ]]; then + s virt-install --noautoconsole --graphics spice,listen=0.0.0.0 \ + --disk=/a/images/win10.qcow2,bus=virtio --vcpus 2 -r 4096 -w bridge=br0 \ + -n win10 --import --os-variant $variant --cpu host-model-only + + s virsh destroy win10 + fi + + if [[ -e /nocow/user/vms/win7.qcow2 ]]; then + # this one hasn\'t had the virtio fix done yet. + s virt-install --noautoconsole --graphics spice,listen=0.0.0.0 \ + --disk=/a/images/win7.qcow2 --vcpus 2 -r 4096 -w bridge=br0 \ + -n win7 --import --os-variant win7 --cpu host-model-only + s virsh destroy win7 + # had a problem with --cpu host, so trying out + # --cpu host-model-only + fi +fi + + +if [[ $HOSTNAME == treetowl ]]; then + pi samba + # note samba re-reads it\'s config every 1 minute + case $distro in + arch) s cp /etc/samba/smb.conf.default /etc/samba/smb.conf ;; + esac + + # add 2 lines after workgroup option + s sed -ri --follow-symlinks '/^\s*encrypt passwords\s*=/d' /etc/samba/smb.conf + s sed -ri --follow-symlinks '/^\s*map to guest\s*=/d' /etc/samba/smb.conf + s sed -i --follow-symlinks 's/\(\s*workgroup\s*=\).*/\1 WORKGROUP\n\tencrypt passwords = yes\n\tmap to guest = bad password/' /etc/samba/smb.conf + # remove default homes section. not sharing that. + s sed -ri --follow-symlinks '/^\s*\[homes\]/,/\s*\[/d' /etc/samba/smb.conf + + if ! grep -xF '[public]' /etc/samba/smb.conf &>/dev/null; then + s tee -a /etc/samba/smb.conf <<'EOF' +[public] + guest ok = yes + read only = no + path = /kr +EOF + fi + + case $distro in + debian|ubuntu) + # systemd claims it generates units from /etc/init.d, but it + # clearly doesn\'t in debian. I have no idea how they are + # related. fuck debian right now. It\'s not documented. samba + # has a systemd init file linked to /dev/null. There\'s this + # https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=769714 which + # claims samba\'s sub-services will be started automatically by + # systemd... it didn\'t on install, wonder if it will on + # boot. It clued me in how to start it manually though. Nothing + # in /usr/share/doc/samba, debian admin guide says nothing about + # any of this. (this is in debian testing as of 4/2016). + + s /etc/init.d/samba start + ;; + arch) + sgo samba + ;; + esac +fi + +tu /etc/hosts <<< "127.0.1.1 $(hostname).lan $(hostname)" + + +######### begin stuff belonging at the end ########## + + +# Apps we want to override others for default file handler: +# simplest way in debian is to just install them last. +simple_packages+=( + mpv +) + +case $distro in + ubuntu|debian) + spa spacefm-gtk3 ;; + arch) + spa spacefm ;; +esac + + +pi "${simple_packages[@]}" + + +if $pending_reboot; then + echo "$0: pending reboot and then finished. doing it now." + s reboot now +else + echo "$0: $(date): ending now)" +fi diff --git a/dsremote b/dsremote new file mode 100755 index 0000000..2557b73 --- /dev/null +++ b/dsremote @@ -0,0 +1,28 @@ +#!/bin/bash -l +# Copyright (C) 2016 Ian Kelling + +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at + +# http://www.apache.org/licenses/LICENSE-2.0 + +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +set -eE -o pipefail +trap 'echo "$0:$LINENO:error: \"$BASH_COMMAND\" returned $?" >&2' ERR + +host=$1 + +if [[ ! $host || $host == -h ]]; then + echo "$0: error: expected 1 arg of hostname" + exit 1 +fi + +rlu $host /a/bin/distro-setup/ +ssh $host /a/bin/distro-setup/distro-begin +ssh $host /a/bin/distro-setup/distro-end diff --git a/dynamic-ip-update.sh b/dynamic-ip-update.sh new file mode 100755 index 0000000..a2da871 --- /dev/null +++ b/dynamic-ip-update.sh @@ -0,0 +1,32 @@ +#!/bin/bash -l + + +# note: in practice, I've not seen my ip address change under comcast +# for over a year. If the internet hadn't mislead me, I wouldn't have +# bothered. + + + + +# based on: https://www.namecheap.com/support/knowledgebase/article.aspx/36/11/how-do-i-start-using-dynamic-dns + +# go to advanced dns, enable the little slider checkbox for dynamic dns, +# add dnynamic dns records for @ and * (not sure * will work, but eh), +# with the initial ip you want. remove any other host records, for example +# the initial default ones. copy the dynamic dns password to /p/dynamic-ip-pass. + +# other articles I found usefull previously, but not the last time +# http://mwholt.blogspot.com/2013/09/how-to-set-up-dynamic-dns-in-5-minutes.html +# https://www.namecheap.com/support/knowledgebase/article.aspx/583/11/how-do-i-configure-ddclient + +ip=`curl -s4 echoip.com` +curl -sS "https://dynamicdns.park-your-domain.com/update?host=@&domain=$HOME_DOMAIN&password=$(cat /p/dynamic-ip-pass)&ip=$ip" > /dev/null + +# an alternative, putting my ip on some known server, +# allows ssh to home if I can access that server: +# ssh -o "ProxyCommand ssh someserver -W desktop:22" desktop + +# ssh root@some_server bash <<'EOF' | log-once dynamic-ip +# sed -i --follow-symlinks '/desktop$/d' /etc/hosts +# echo "${SSH_CLIENT%% *} desktop" >> /etc/hosts +# EOF diff --git a/input-setup b/input-setup new file mode 100755 index 0000000..2f595f3 --- /dev/null +++ b/input-setup @@ -0,0 +1,139 @@ +#!/bin/bash -l +# Copyright (C) 2016 Ian Kelling + +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at + +# http://www.apache.org/licenses/LICENSE-2.0 + +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +#set -x +mi() { + xinput --get-feedbacks "$1" | grep "threshold" + xinput --get-feedbacks "$1" | grep "accelNum\|accelDenom" + xinput --list-props "$1" | grep "Device Accel Profile\|Device Accel Constant Deceleration\|Device Accel Velocity Scaling" +} +ms() { + xinput --set-ptr-feedback "$1" $2 ${3%/*} ${3#*/} + xinput --set-prop "$1" 'Device Accel Profile' $4 + xinput --set-prop "$1" 'Device Accel Constant Deceleration' $5 + xinput --set-prop "$1" 'Device Accel Velocity Scaling' $6 + mi "$1" +} +set_device_id() { + if device_id=$(xinput --list | grep -m 1 "$1"); then + device_id=${device_id##*id=} + device_id=${device_id%%[[:space:]]*} + else + return 1 + fi +} + +case $HOSTNAME in + tp|x2) + # original saved with: xkbcomp $DISPLAY /a/c/stretch-11-2016.xkb + xkbcomp /a/c/x2.xkb $DISPLAY + ;; + treetowl*|iank-dev|frodo) + # todo, differentiate for work pc + #/a/bin/radl + if [[ -z $DISPLAY ]]; then + echo "error. empty DISPLAY var" >> /a/gdm-keyboard-error.log + else + xkbcomp /a/c/.Xkeymap $DISPLAY + + xkbset m # setup mouse keys, which I have altered + + xkbset exp =m # stop mousekeys expiring after a timeout + + xset r rate 200 13 # decrease rate delay + cd / # so xbindkeys does not hold open mountpoints + xbindkeys # configured to grab left and right scroll button presses + fi + + + #right scroll wheel, change from button 4 & 5 to 13 and 14. + # also changes the middle click to 12, even though I'm not using it anymore + if set_device_id "04d9:048e"; then + xinput --set-button-map "$device_id" 1 12 3 13 14 6 7 + fi + + ms 'Kensington Kensington Slimblade Trackball' 100 4000/1 7 6.5 1.5 + xinput --set-button-map 'Kensington Kensington Slimblade Trackball' 0 0 0 4 5 6 7 0 9 10 11 12 + + # razer naga middle mouse stopped working. it's settings were: + #ms "$device_id" 100 1000/1 7 4 1 + + if set_device_id "SteelSeries World of Warcraft MMO Gaming Mouse"; then + ms "$device_id" 100 1000/1 7 4 1 + #ms "$device_id" 1 7/2 2 2.5 2 + + # makes it compatible with what windows sends using synergy + xinput --set-button-map "$device_id" 1 2 3 4 5 6 7 6 7 10 11 12 13 14 15 + + # under the new "improved" libinput, the mouse speed/accel has + # changed all around and is much more limited. + # Other xinput commands will fail and this will succeed. + xinput --set-prop "$device_id" "libinput Accel Speed" '.8' + fi + + + # disable the mouse movements of my mouse-wheel only mouse + if set_device_id "USB Optical Mouse"; then + xinput --set-prop "$device_id" 'Device Accel Constant Deceleration' 10000 + # 12 is to effectively disable the middle click button + xinput --set-button-map "$device_id" 1 12 3 10 11 6 7 + fi + . /a/bin/bash_unpublished/duplicity-gpg-agent-setup + ;; + frodo*) + ;; +esac + +# for desktop and htpc +set_device_id "Logitech Unifying Device" +xinput --set-prop "$device_id" 'Evdev Middle Button Emulation' 1 + + +#ms 'Kensington Kensington Slimblade Trackball' 1 7/2 2 4 5 = 2.01 +# ms 9 10 20/1 6 4 .2 +#ms 9 10 35/1 6 5 .2 = 1.82 +#ms 9 10 1200/1 7 5 .2 = 1.82 after a practice round +# +#ms 9 10 140/1 6 7 .1 = 2.0 after a practice round. +#feels like the slow is finally too slow, and fast too fast +# ms 9 10 50/1 6 6 .2 = 1.83 after +# ms 9 10 90/1 6 6 .1 = 1.86 +#ms 9 10 3000/1 7 6 .1 = 1.81 +#ms 9 1 15/1 3 6 3 = 2.0 +#ms 9 1 10/1 3 4 2 = 1.91 +#ms 9 1 8/1 3 5 4 = 1.98 +#ms 9 1 10/2 2 5 2.5 = 1.99 +#ms 9 100 3000/1 7 5 1 = 1.86 +#ms 9 100 1500/1 7 5 1.5 = 1.87 +#ms 9 100 2200/1 7 6 1.5 = 1.81 +#ms 9 100 2200/1 7 5 1.5 = 1.83 +#ms 9 100 3000/1 7 6 1.5 = 1.88 +#ms 9 1 100/1 4 6 15 = 1.85 +#ms 9 100 3000/1 7 7 1.5 = 1.85, but had a 1.76 on first try... +# ms 9 100 3500/1 7 7 1.5 = 1.73, 1.68 a1 1.74, 1.83, 1.75, 1.78, 1.76 +# ms 9 100 4000/1 7 7 1.5 = 1.80 +# ms 9 100 4000/1 7 8 1.5 = 1.78 +# ms 9 100 4000/1 7 8 1.8 = 1.88 +# ms 9 100 4800/1 7 8 1.5 = 1.85 +# ms 9 100 4000/1 7 6.5 1.5 = 1.83. stickig with this, upped speeds a bit to make it more practical for normal windows +# new day +# ms 9 100 3500/1 7 7 1.5 = 1.92 +# ms 9 1 7/2 2 5 5 = 1.9 +# ms 9 100 3500/1 7 7 1.5 = 1.62 +# felt like having it a bit faster for a while, but switched back +#ms 'Kensington Kensington Slimblade Trackball' 100 3500/1 7 5 1.5 + + +#set +x diff --git a/install-my-scripts b/install-my-scripts new file mode 100755 index 0000000..42a6d2d --- /dev/null +++ b/install-my-scripts @@ -0,0 +1,34 @@ +#!/bin/bash +# Copyright (C) 2016 Ian Kelling + +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at + +# http://www.apache.org/licenses/LICENSE-2.0 + +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +# Running these files directly won't be good since we are +# unmounting the volume they live on. +# This never really get's run, since we normally only +# seed these files to other hosts using btrbk-run. + + +set -eE -o pipefail +trap 'echo "$0:$LINENO:error: \"$BASH_COMMAND\" returned $?" >&2' ERR + +[[ $EUID == 0 ]] || exec sudo -E "$BASH_SOURCE" "$@" + +x="$(readlink -f "$BASH_SOURCE")"; cd ${x%/*} + + +e() { echo "$*"; "$@"; } + +# scripts that would interfere with unmounting /a, put them elsewhere +e install mail-cert-cron mount-latest-subvol check-subvol-stale /usr/local/bin diff --git a/keyscript-off b/keyscript-off new file mode 100755 index 0000000..c4ac8d5 --- /dev/null +++ b/keyscript-off @@ -0,0 +1,43 @@ +#!/bin/bash +# Copyright (C) 2016 Ian Kelling + +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at + +# http://www.apache.org/licenses/LICENSE-2.0 + +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +set -eE -o pipefail +trap 'echo "$0:$LINENO:error: \"$BASH_COMMAND\" returned $?"' ERR + +[[ $EUID == 0 ]] || exec sudo "${BASH_SOURCE}" "$@" +if [[ $- != *i* ]]; then + exec &>/var/log/keyscript-off.log + echo "$0: starting. $(date)" +fi + +sed="sed --follow-symlinks" + +if [[ $($sed -rn 's/^ID=(.*)/\1/p' /etc/os-release) == arch ]]; then + if grep -q '^\s*FILES=' /etc/mkinitcpio.conf; then + $sed -ri 's/^\s*FILES=/#\0/' /etc/mkinitcpio.conf # comment out + mkinitcpio -p linux + fi +else + x=/root/keyscript + if grep -q "${x}," /etc/crypttab; then + $sed -i "s#${x},#${x}-manual,#" /etc/crypttab + update-initramfs -u + fi +fi + +# switch to easy or hard pass which is the same as luks +f=/q/root/shadow/traci +[[ $HOSTNAME != tp ]] || usermod -p "$(cat $f)" ian +echo "$0: finished. $(date)" diff --git a/keyscript-on b/keyscript-on new file mode 100755 index 0000000..54a655c --- /dev/null +++ b/keyscript-on @@ -0,0 +1,46 @@ +#!/bin/bash +# Copyright (C) 2016 Ian Kelling + +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at + +# http://www.apache.org/licenses/LICENSE-2.0 + +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +set -eE -o pipefail +trap 'echo "$0:$LINENO:error: \"$BASH_COMMAND\" returned $?"' ERR + +[[ $EUID == 0 ]] || exec sudo "${BASH_SOURCE}" "$@" +if [[ $- != *i* ]]; then + exec &>>/var/log/keyscript-on.log + echo "$0: starting. $(date)" +fi +rootn=1 + +sed="sed --follow-symlinks" + +if [[ ! -e /tmp/keyscript-off ]]; then + if [[ $($sed -rn 's/^ID=(.*)/\1/p' /etc/os-release) == arch ]]; then + if ! grep -q '^\s*FILES=' /etc/mkinitcpio.conf; then + $sed -ri 's/^#(\s*FILES=.*)/\1/' /etc/mkinitcpio.conf # uncomment + mkinitcpio -p linux + fi + else + x=/root/keyscript + if grep -q "${x}-manual," /etc/crypttab; then + $sed -i "s#${x}-manual,#${x},#" /etc/crypttab + update-initramfs -u + fi + fi +fi +# switch to easy or hard login pass which is the same as luks +f=/q/root/shadow/traci-simple +[[ $HOSTNAME != tp ]] || usermod -p "$(cat $f)" ian + +echo "$0: finished. $(date)" diff --git a/mail-cert-cron b/mail-cert-cron new file mode 100644 index 0000000..64b22eb --- /dev/null +++ b/mail-cert-cron @@ -0,0 +1,27 @@ +#!/bin/bash +# Copyright (C) 2016 Ian Kelling + +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at + +# http://www.apache.org/licenses/LICENSE-2.0 + +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +set -eE -o pipefail +trap 'echo "$0:$LINENO:error: \"$BASH_COMMAND\" returned $?" >&2' ERR + +[[ $EUID == 0 ]] || exec sudo "$BASH_SOURCE" "$@" + +source /a/bin/bash_unpublished/source-semi-priv +if [[ $HOSTNAME == $MAIL_HOST ]]; then + local_mx=mail.iankelling.org + rsync_common="rsync -ogt --chown=root:Debian-exim --chmod=640 root@li:/p/c/machine_specific/li/webservercerts/$local_mx-" + ${rsync_common}chained.pem /etc/exim4/exim.crt + ${rsync_common}domain.key /etc/exim4/exim.key +fi diff --git a/mail-route b/mail-route new file mode 100755 index 0000000..dc74c7b --- /dev/null +++ b/mail-route @@ -0,0 +1,75 @@ +#!/bin/bash +# Copyright (C) 2016 Ian Kelling + +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at + +# http://www.apache.org/licenses/LICENSE-2.0 + +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +[[ $EUID == 0 ]] || exec sudo "$BASH_SOURCE" "$@" + +source /a/bin/errhandle/errcatch-function +source /a/bin/errhandle/errallow-function +source /a/bin/errhandle/bash-trace-function +errcatch + +usage() { + cat <<'EOF' +Usage: mail-route start|stop|show +EOF + exit $1 +} + +if (( $# != 1 )); then + usage 1 +fi +case $1 in + start) + iptables_op=-A + ip_op=add + e() { "$@"; } + ;; + stop) + iptables_op=-D + ip_op=del + e() { "$@" || printf "maybe ok failure: %s\n" "$*"; } + ;; + show) + e() { printf "${0##*/}: %s\n" "$*"; "$@"; } + e iptables -t mangle -S + e iptables -t nat -S + e ip rule + e ip route show table 1 + exit 0 + ;; + *) + usage 1 + ;; +esac + + +# note, something like this does not work for packets which +# exim is replying to. I don't know why. +#iptables -t mangle -A OUTPUT -m owner --uid-owner Debian-exim -j MARK --set-mark 0x1 + +# match source or dest port. when we send to 25, it picks a random high port as +# the source. + +for port in 25 143; do # smtp and imap. + e iptables -t mangle $iptables_op \ + OUTPUT -m tcp -p tcp -m multiport --ports $port -j MARK --set-mark 0x1 +done +e iptables -t nat $iptables_op POSTROUTING -o tun0 -m mark --mark 0x1 -j SNAT --to-source 10.8.0.4 +e ip rule $ip_op fwmark 1 table 1 +# note, this rule does not persist when the tun interface is deleted +e ip route $ip_op default via 10.8.0.1 table 1 +e ip route $ip_op 192.168.1.0/24 via 192.168.1.1 dev br0 table 1 + +exit 0 diff --git a/mail-setup b/mail-setup new file mode 100755 index 0000000..7921a22 --- /dev/null +++ b/mail-setup @@ -0,0 +1,613 @@ +#!/bin/bash -l +# Copyright (C) 2016 Ian Kelling + +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at + +# http://www.apache.org/licenses/LICENSE-2.0 + +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +set -eE -o pipefail +trap 'echo "$0:$LINENO:error: \"$BASH_COMMAND\" returned $?" >&2' ERR + +####### begin perstent password instructions ###### +# # exim passwords: +# # for hosts which have all private files I just use the same user +# # for other hosts, each one get\'s their own password. +# # for generating secure pass, and storing for server too: +# # user=USUALLY_SAME_AS_HOSTNAME +# user=li +# f=$(mktemp) +# apg -m 50 -x 70 -n 1 -a 1 -M CLN >$f +# s sed -i "/^$user:/d" /p/c/filesystem/etc/exim4/passwd +# echo "$user:$(mkpasswd -m sha-512 -s <$f)" >>/p/c/filesystem/etc/exim4/passwd +# echo "mail.iankelling.org:$user:$(<$f)" >> /p/c/machine_specific/$user/filesystem/etc/mailpass +# # then run this script, or part of it which uses /etc/mailpass + +# # dovecot password, i just need 1 as I\'m the only user +# mkdir /p/c/filesystem/etc/dovecot +# echo "ian:$(doveadm pw -s ssha256)::::::" >/p/c/filesystem/etc/dovecot/users +# conflink + + + +# # for ad-hoc testing of some random new host sending mail: +# user=li # client host username & hostname +# f=$(mktemp) +# apg -m 50 -x 70 -n 1 -a 1 -M CLN >$f +# s sed -i "/^$user:/d" /etc/exim4/passwd +# echo "$user:$(mkpasswd -m sha-512 -s <$f)" | s tee -a /etc/exim4/passwd +# echo "mail.iankelling.org:$user:$(<$f)" | ssh root@$user dd of=/etc/exim4/passwd.client +####### end perstent password instructions ###### + + +####### begin persistent dkim/dns instructions ######### +# # Remove 1 level of comments in this section, set the domain var +# # for the domain you are setting up, then run this and copy dns settings +# # into dns. +# domain=iankelling.org +# c /p/c/filesystem/etc/exim4 +# # this has several bugs addressed in comments, but it was helpful +# # https://debian-administration.org/article/718/DKIM-signing_outgoing_mail_with_exim4 + +# openssl genrsa -out $domain-private.pem 2048 -outform PEM +# openssl rsa -in $domain-private.pem -out $domain.pem -pubout -outform PEM +# # selector is needed for having multiple keys for one domain. +# # I dun do that, so just use a static one: li +# echo "txt record name: li._domainkey.$domain" +# # Debadmin page does not have v=, fastmail does, and this +# # says it\'s recommended in 3.6.1, default is DKIM1 anyways. +# # https://www.ietf.org/rfc/rfc6376.txt +# # Join and print all but first and last line. +# # last line: swap hold & pattern, remove newlines, print. +# # lines 2+: append to hold space +# echo "txt record contents:" +# echo "v=DKIM1; k=rsa; p=$(sed -n '${x;s/\n//gp};2,$H' $domain.pem)" +# chmod 644 $domain.pem +# chmod 640 $domain-private.pem +# # in conflink, we chown these to group debian +# conflink +# # selector was also put into /etc/exim4/conf.d/main/000_localmacros, +# # via the mail-setup scripts + +# # 2017-02 dmarc policies: +# # yahoo: p=reject, hotmail: p=none, gmail: p=none, fastmail none for legacy reasons +# # gmail will be changing to p=reject, which is expected to cause problems +# # with a few old mailing lists, copying theirs for now. +# echo "dmarc dns, name: _dmarc value: v=DMARC1; p=none; rua=mailto:mailauth-reports@$domain" + +# # 2017-02 spf policies: +# # google ~all, hotmail -all, yahoo: ?all, fastmail ?all +# # i include fastmail\'s settings, per their instructions, +# # and follow their policy. In mail in a box, or similar instructions, +# # I\'ve seen recommended to not use a restrictive policy. +# echo "spf dns: name is empty, value: v=spf1 a include:spf.messagingengine.com ?all" + +# # to check if dns has updated, you do +# host -a mesmtp._domainkey.$domain + +# # mx records, +# # setting it to iankelling.org would work the same, but this +# # is more flexible, I could change where mail.iankelling.org pointed. +# cat <<'EOF' +# mx records, 2 records each, for * and empty domain +# pri 10 mail.iankelling.org +# pri 20 in1-smtp.messagingengine.com +# pri 30 in2-smtp.messagingengine.com +# EOF +####### end persistent dkim instructions ######### + + +# misc exim notes: +# useful exim docs: +# /usr/share/doc/exim4-base/README.Debian.gz +# /usr/share/doc/exim4-base/spec.txt.gz + +# routers, transports, and authenticators are sections, and you define +# driver instances in those sections, and the manual calls them driver +# types but there is also a more specific "type" of driver, which is specified +# with the driver = some_module setting in the driver. + +# the driver option must precede and private options (options that are +# specific to that driver), so follow example of putting it at beginning. + +# The full list of option settings for any particular driver instance, +# including all the defaulted values, can be extracted by making use of +# the -bP command line option. + +# exim clear out message queue. as root: +# adapted from somewhere on stackoverflow. +# ser stop exim4; sleep 1; exim -bp | exiqgrep -i | xargs exim -Mrm; ser start exim4 + +# fastmail has changed their smtp server, but the old one still works, +# I see no reason to bother changing. +# New one is smtp.fastmail.com + +# test delivery & rewrite settings: +#exim4 -bt ian@localhost + + +type=$1 +postfix() { [[ $type == postfix ]]; } +exim() { [[ $type == exim4 ]]; } +if ! exim && ! postfix; then + echo "$1: error: expected exim4 or postfix as first arg" + exit 1 +fi + + +local_mx=mail.iankelling.org + +host=$local_mx +relayhost="[$host]:25" # postfix +smarthost="$host::25" # exim + + +# this was for when I used the exim config type +# "mail sent by smarthost; received via SMTP or fetchmail" +# if [[ $HOSTNAME == $MAIL_HOST ]]; then +# host=mail.messagingengine.com +# relayhost="[$host]:587" # postfix +# smarthost="$host::587" # exim +# fi + +forward=ian@$local_mx + + +if [[ $HOSTNAME == $MAIL_HOST ]]; then + # if we are MAIL_HOST, exim config sets up an /etc/alias from + # root to the postmaster, which i config to ian, as long as there + # exists an entry for root, or there was no preexisting aliases file. + # based on the postinst file. + s rm -f /etc/aliases +else + # linode image has a root alias, I think it might override our .forward + sudo sed -i '/^root:/d' /etc/aliases + s newaliases + + # background: This also works instead of ~/.forward + # s sed -i --follow-symlinks '/^root/d' /etc/aliases ||: + #echo "root: $HOSTNAME@$SOME_DOMAIN" | s tee -a /etc/aliases + # this can\'t be a symlink and has permission restrictions + # it might work in /etc/aliases, but this seems more proper. + e $forward > ~/.forward + e $forward | s tee /root/.forward + # 644 is required. shouldn\'t need changing, but set it just in case. + s chmod 644 ~/.forward /root/.forward +fi + +# offlineimap uses this too, it is much easier to use one location than to +# condition it\'s config and postfix\'s config +case $distro in + fedora) s lnf -T ca-certificates.crt /etc/ssl/ca-bundle.trust.crt ;; + *) : +esac + +if postfix; then + # dunno why, but debian installed postfix with builddep emacs + # but I will just explicitly install it here since + # I use it for sending mail in emacs. + if isdeb; then + s debconf-set-selections </dev/null + done + s postmap hash:/etc/postfix/sasl_passwd + s service postfix reload + +else # exim. has debian specific stuff for now + + + # wording of question from dpkg-reconfigure exim4-config + # 1. internet site; mail is sent and received directly using SMTP + # 2. mail sent by smarthost; received via SMTP or fetchmail + # 3. mail sent by smarthost; no local mail + # 4. local delivery only; not on a network + # 5. no configuration at this time + # + # Note, I have used option 2 in the past for receiving mail + # from lan hosts, sending external mail via another smtp server. + # + # Note, other than configtype, we could set all the options in + # both types of configs without harm, they would either be + # ignored or be disabled by other settings, but the default + # local_interfaces definitely makes things more secure. + + # most of these settings get translated into settings + # in /etc/exim4/update-exim4.conf.conf + # mailname setting sets /etc/mailname + + s debconf-set-selections </dev/null <<'EOF' +MAIN_TLS_ENABLE = true + +DKIM_CANON = relaxed +DKIM_SELECTOR = li + +# from comments in +# https://debian-administration.org/article/718/DKIM-signing_outgoing_mail_with_exim4 + +# The file is based on the outgoing domain-name in the from-header. +DKIM_DOMAIN = ${lc:${domain:$h_from:}} +# sign if key exists +DKIM_PRIVATE_KEY= ${if exists{/etc/exim4/${dkim_domain}-private.pem} {/etc/exim4/${dkim_domain}-private.pem}} + + +# failing message on mail-tester.com: +# We check if there is a server (A Record) behind your hostname treetowl. +# You may want to publish a DNS record (A type) for the hostname treetowl or use a different hostname in your mail software +# https://serverfault.com/questions/46545/how-do-i-change-exim4s-primary-hostname-on-a-debian-box +# and this one seemed appropriate from grepping config +MAIN_HARDCODE_PRIMARY_HOSTNAME = li.iankelling.org + +# normally empty, I set this so I can set the envelope address +# when doing mail redelivery to invoke filters +MAIN_TRUSTED_GROUPS = ian + +LOCAL_DELIVERY = dovecot_lmtp + +CHECK_RCPT_LOCAL_ACL_FILE = /etc/exim4/rcpt_local_acl +EOF + + + s dd of=/etc/systemd/system/offlineimapsync.timer <<'EOF' +[Unit] +Description=Run offlineimap-sync once every 5 mins + +[Timer] +OnCalendar=*:0/5 + +[Install] +WantedBy=timers.target +EOF + + s dd of=/etc/systemd/system/offlineimapsync.service <<'EOF' +[Unit] +Description=Offlineimap sync +After=multi-user.target + +[Service] +User=ian +Type=oneshot +ExecStart=/a/bin/log-quiet/sysd-mail-once offlineimap-sync /a/bin/distro-setup/offlineimap-sync +EOF + s systemctl daemon-reload + s systemctl enable offlineimapsync.timer + s systemctl start offlineimapsync.timer + + else # $HOSTNAME != $MAIL_HOST + s systemctl disable offlineimapsync.timer &>/dev/null ||: + s systemctl stop offlineimapsync.timer &>/dev/null ||: + # + # + # would only exist because I wrote it i the previous condition, + # it\'s not part of exim + s rm -f $exim_main_dir/000_localmacros + s debconf-set-selections </dev/null; then + # gotta remove this, otherwise the set-selections are completely + # ignored. It woulda been nice if this was documented somewhere! + s rm -f /etc/exim4/update-exim4.conf.conf + s dpkg-reconfigure -u -fnoninteractive exim4-config + fi + # light version does not have sasl auth support. + pi exim4-daemon-heavy spamassassin + + ##### begin spamassassin config + ser enable spamassassin + # per readme.debian + s sed -i '/^\s*CRON\s*=/d' /etc/default/spamassassin + s tee -a /etc/default/spamassassin <</dev/null + done + # end setup passwd.client + + # https://blog.dhampir.no/content/make-exim4-on-debian-respect-forward-and-etcaliases-when-using-a-smarthost + # i only need .forwards, so just doing that one. + cd /etc/exim4/conf.d/router + a=userforward + b=${a}_higher_priority + tmp=$(mktemp) + of=175_$b + # sed to make the router name unique + sed -r s/^\\S+:/$b:/ 600_exim4-config_$a | s dd of=$tmp 2>/dev/null + if ! diff -q $tmp $of &>/dev/null; then + s dd if=$tmp of=$of >/dev/null + fi + + + ser restart exim4 + +fi + + + + + +# based on http://www.postfix.org/qmgr.8.html and my notes in gnus +dir=/nocow/$type +sdir=/var/spool/$type +if [[ $(readlink -f $sdir) != $dir ]]; then + ser stop $type + if [[ ! -e $dir && -d $sdir ]]; then + s mv $sdir $dir + fi + s lnf -T $dir $sdir +fi + +sgo $type + + +# if I wanted the from address to be renamed and sent to a different address, +# echo "sdx@localhost development@localhost" | sudo dd of=/etc/postfix/recipient_canonical +# sudo postmap hash:/etc/postfix/recipient_canonical +# sudo service postfix reload diff --git a/maru-init b/maru-init new file mode 100755 index 0000000..8f4ab55 --- /dev/null +++ b/maru-init @@ -0,0 +1,57 @@ +#!/bin/bash +# Copyright (C) 2017 Ian Kelling + +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at + +# http://www.apache.org/licenses/LICENSE-2.0 + +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +set -eE -o pipefail +trap 'echo "$0:$LINENO:error: \"$BASH_COMMAND\" returned $?" >&2' ERR + + +usage() { + cat <>/etc/sudoers +echo n5 >/etc/hostname +sed -i '/^127\.0\.1\.1/d' /etc/hosts +echo "127.0.1.1 n5.lan n5" >>/etc/hosts +hostname -F /etc/hostname + +kill $(pgrep -U maru) +usermod -l ian -m -d /home/ian maru +groupmod -n ian maru +useradd -m -s /bin/bash traci +EOF + +# then do myunison n5, +# then do conflink. diff --git a/mount-latest-remote b/mount-latest-remote new file mode 100755 index 0000000..cd04faf --- /dev/null +++ b/mount-latest-remote @@ -0,0 +1,37 @@ +#!/bin/bash +# Copyright (C) 2016 Ian Kelling + +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at + +# http://www.apache.org/licenses/LICENSE-2.0 + +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +set -eE -o pipefail +trap 'echo "$0:$LINENO:error: \"$BASH_COMMAND\" returned $?" >&2' ERR + +script_dir=$(dirname $(readlink -f "$BASH_SOURCE")) + +if [[ ! $@ ]]; then + echo "mount-latest-remote: error: expected 1 or more host arguments" + exit 1 +fi + +for tg; do + scp $script_dir/{mount-latest-subvol,check-subvol-stale} \ + root@$tg:/usr/local/bin + if ! ssh root@$tg bash <<'EOF' +set -e +chmod +x /usr/local/bin/{mount-latest-subvol,check-subvol-stale} +/usr/local/bin/mount-latest-subvol +EOF + then + echo "$0: warning: failed mount-latest-subvol on $tg" + fi +done diff --git a/mount-latest-subvol b/mount-latest-subvol new file mode 100644 index 0000000..c43bb58 --- /dev/null +++ b/mount-latest-subvol @@ -0,0 +1,212 @@ +#!/bin/bash +# Copyright (C) 2016 Ian Kelling +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# usage: mount-latest-subvol + +[[ $EUID == 0 ]] || exec sudo -E "$BASH_SOURCE" "$@" + +errcatch() { + set -E; shopt -s extdebug + _err-trap() { + err=$? + exec >&2 + set +x + echo "${BASH_SOURCE[1]}:${BASH_LINENO[0]}:in \`$BASH_COMMAND' returned $err" + bash-trace 2 + echo "$0: exiting with code $err" + exit $err + } + trap _err-trap ERR + set -o pipefail +} +bash-trace() { + local -i argc_index=0 arg frame i start=${1:-1} max_indent=8 indent + local source + local extdebug=false + if [[ $(shopt -p extdebug) == *-s* ]]; then + extdebug=true + fi + + for ((frame=0; frame < ${#FUNCNAME[@]}-1; frame++)); do + argc=${BASH_ARGC[frame]} + argc_index+=$argc + ((frame < start)) && continue + if (( ${#BASH_SOURCE[@]} > 1 )); then + source="${BASH_SOURCE[frame+1]}:${BASH_LINENO[frame]}:" + fi + indent=$((frame-start+1)) + indent=$((indent < max_indent ? indent : max_indent)) + printf "%${indent}s↳%sin \`%s" '' "$source" "${FUNCNAME[frame]}" + if $extdebug; then + for ((i=argc_index-1; i >= argc_index-argc; i--)); do + printf " %s" "${BASH_ARGV[i]}" + done + fi + echo \' + done +} +errcatch + +tu() { + while read -r line; do + file="$1" + grep -xFq "$line" "$file" || tee -a "$file"<<<"$line" + done +} +e() { printf "%s\n" "$*"; "$@"; } +mnt() { + dir=$1 + if ! mountpoint $dir &>/dev/null; then + mkdir -p $dir + e mount $dir + fi +} + +ret=0 + +##### begin setup fstab for subvols we care about ###### +first_root_crypt=$(awk '$2 == "/" {print $1}' /etc/mtab) +tu /etc/fstab </dev/null; then + continue + fi + + + ##### begin building up list of bind mounts ###### + binds=() # list of bind mounts + roots=($d) # list of bind mounts, plus the original mount + while true; do + new_roots=() + for r in ${roots[@]}; do + # eg. when r=/q/p, for lines like + # /q/p /p none bind 0 0 + # output /p + new_roots+=($(sed -rn "s#^$r/\S+\s+(\S+)\s+none\s+bind\s.*#\1#p" /etc/fstab)) + done + (( ${#new_roots} )) || break + binds+=(${new_roots[@]}) + roots=( ${new_roots[@]} ) + done + ##### end building up list of bind mounts ###### + + + # if latest is already mounted, make sure binds are mounted and move on + if e check-subvol-stale $d; then + for b in ${binds[@]}; do + mnt $b + done + continue + fi + + last_snap=$(&2' ERR + +if [[ $EUID != 0 ]]; then s=sudo; fi + +$s touch /tmp/keyscript-off +$s poweroff diff --git a/offlineimap-sync b/offlineimap-sync new file mode 100755 index 0000000..a179b36 --- /dev/null +++ b/offlineimap-sync @@ -0,0 +1,33 @@ +#!/bin/bash + +set -eE -o pipefail +trap 'echo "$0:$LINENO:error: \"$BASH_COMMAND\" returned $?" >&2' ERR + +offlineimap -u quiet +shopt -s nullglob + +omv() { # offlineimap mv + src="$1" + dst="$2" + found_files=false + for x in new cur; do + files=("$src_base"/"$src"/$x/*) + if [[ $files ]]; then + found_files=true + mv "${files[@]}" /m/md/"$dst"/$x + fi + done +} + +src_base=/m/offlineimap +omv "Sent Items" "Sent" +omv INBOX offlineimaptmp +src_base=/m/md +if $found_files; then + sieve-filter -eW ~/sieve/main.sieve offlineimaptmp &>/dev/null + # the default folder is INBOX for anything leftover + omv offlineimaptmp INBOX + # remove messages from remote host + offlineimap -u quiet + mu index &>/dev/null ||: +fi diff --git a/phabricator-setup b/phabricator-setup new file mode 100755 index 0000000..7c63e4f --- /dev/null +++ b/phabricator-setup @@ -0,0 +1,358 @@ +#!/bin/bash -l +# Copyright (C) 2016 Ian Kelling + +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at + +# http://www.apache.org/licenses/LICENSE-2.0 + +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + + +# Automated phabricator setup. Not currently using it, +# but it worked last time I tried it. + +set -eE -o pipefail +trap 'echo "$0:$LINENO:error: \"$BASH_COMMAND\" returned $?"' ERR + +set -x + + +# lj is test server +case $HOSTNAME in + lj) + domain=phab.iank.bid + alt_domain=fastmail.wiki + ;; + lk) + domain=phab.iankelling.org + alt_domain=iankellingusercontent.org + ;; +esac + + +pass=`cat /p/c/machine_specific/$HOSTNAME/phabricator_admin` +webroot=/usr/share/phabricator/webroot +user=iank +name="Ian Kelling" +email=ian@iankelling.org +ssh_port=222 + +fbin() { bin=$1; shift; sudo /usr/share/phabricator/bin/$bin "$@"; } +fsetd() { fbin config set --database "$@"; } + +# phabricator complained about wanting arcanist first +pi arcanist/unstable mercurial + +# duplicated in mediawiki setup. todo fix that. +s DEBIAN_FRONTEND=noninteractive pi mysql-server +cd # mysql_secure_installation writes some temp files to the current dir, +# so we need to make sure it's writable. +if echo exit|mysql -u root -p"$dbpass"; then + echo -e "$dbpass\nn\n\n\n\n" | mysql_secure_installation +else + echo -e "\n\n$dbpass\n$dbpass\n\n\n\n\n" | mysql_secure_installation +fi + +mysql -u root -p$dbpass < + Require all granted + +EOF +done + + +# Before I figured out how to setup the admin in the script, +# this would limit the site to localhost, +# and access it through an ssh tunnel until its secure. +#phab-site -p 127.0.0.1:443 + +# settings are stored in conf/local/local.json. +# some settings could also be stored in the database with +# --database arg. database has higher priority than +# the config file. + +# if you need to restart phabricator, just ser restart apache2 +# https://secure.phabricator.com/book/phabricator/article/restarting/ + +# to reset things, you can do. +# fbin storage destroy; pu phabricator; phab-sel; pi phabricator/unstable +# # but under debian, prolly better to purge, cause db gets created on install + + +# On first run went to the website, registered manually, then +# went through the gui setup items to get the configuration below. + + +#expect "*" +#sleep 1 + +# expect's exits with 0 by default on timeout of an expect command. +# You can modify this, but it was simpler to use an irregular code to detect +# actual success. +sudo expect -d <<()~*:\"\"&^'" +# default is 128M. recommended starting point is 40% of ram. +setd innodb_buffer_pool_size 1600M + +# this files stopwork, and min_word_len +mysql -u root -p$dbpass <<'EOF' +REPAIR TABLE phabricator_search.search_documentfield; +EOF + +fsetd pygments.enabled true +fbin config set security.alternate-file-domain https://$alt_domain + +setini opcache.validate_timestamps '"0"' opcache /etc/php5/apache2/php.ini +setini post_max_size 100M PHP /etc/php5/apache2/php.ini + +fsetd metamta.default-address phabricator@$domain +fsetd metamta.domain $domain + + +ser restart mysql + +# Not sure if this is needed. while developing this script, mysql went down +# for a bit and the daemons died. + + +# todo, setup inbound email: +# https://secure.phabricator.com/book/phabricator/article/configuring_inbound_email/ + + +# https://secure.phabricator.com/book/phabricator/article/diffusion_hosting/ +# unmatchable password, allows login only via ssh, sudo, etc. +# this is standard. +# I tried having no home dir, (-d /nonexistent), +# but I got an error message on test sshing, +sudo useradd -p '*' -m --system -s /bin/sh vcs || [[ $? == 9 ]] + +# you'd think the debian package would set this. todo: check on a fresh +# machine +fbin config set phd.user phabricator +fbin config set diffusion.ssh-user vcs + +option="ALL=(phabricator) SETENV: NOPASSWD:" +www_files=$(which git hg|sed ':a;N;s/\n/, /;ta') +vcs_files=$(which git git-upload-pack git-receive-pack hg|sed ':a;N;s/\n/, /;ta') +[[ $www_files && $vcs_files ]] || exit 1 +www_files="$www_files, /usr/lib/git-core/git-http-backend" +sudo dd of=/etc/sudoers.d/phabricator </tmp/plog 2>&1 +# This script executes as the vcs user +if [ "$1" != vcs ]; then exit 1; fi +exec "/usr/share/phabricator/bin/ssh-auth" $@ +EOF +sudo chmod 755 $file + +sudo dd of=/etc/ssh/sshd_config.phabricator </src/aphront/storage/connection/mysql/AphrontBaseMySQLDatabaseConnection.php:306] +# arcanist(), phabricator(), phutil() + +s usermod -a -G vcs www-data +s usermod -a -G vcs ian +s usermod -a -G vcs phabricator +s chown root:vcs /usr/share/phabricator/conf/local/local.json +fbin config set diffusion.ssh-port $ssh_port + +fsetd policy.allow-public true + +sgo phabricator-ssh + +ser restart apache2 +sgo phabricator + + +# todo, finish next steps here: +# notably, backup/restore +# https://secure.phabricator.com/book/phabricator/article/configuration_guide/ + + +fbin auth recover iank + +cat </dev/null # too verbose by default +rsync -r --delete root@li:{/home/ian/dump,/home/pumpio/pumpdata} /w/backup/pump diff --git a/radicale-setup b/radicale-setup new file mode 100755 index 0000000..b885f36 --- /dev/null +++ b/radicale-setup @@ -0,0 +1,76 @@ +#!/bin/bash -l + +[[ $EUID == 0 ]] || exec sudo -E "$BASH_SOURCE" "$@" + + +# davdroid setup instructions at the bottom + +# main docs: +# http://radicale.org/user_documentation/ +# https://davdroid.bitfire.at/configuration/ + + +# created password file with: +# htpasswd -c /etc/nginx/caldav/htpasswd ian + + +# python-dulwich, to track changes with git, per +# http://radicale.org/user_documentation/#idgit-support +pi nginx python-dulwich radicale + +# I moved /var/lib/radicale after it's initialization. +# I did a sudo -u radicale git init in the collections subfolder +# after it gets created, per the git docs. +lnf -T /o/radicale /var/lib/radicale + +# from https://www.williamjbowman.com/blog/2015/07/24/setting-up-webdav-caldav-and-carddav-servers/ + +# more config is for li in distro-end + +# coment in this file says this is needed for it to run on startup +sed -ri 's/^\s*#+\s*(ENABLE_RADICALE\s*=\s*yes\s*)/\1/' /etc/default/radicale + +setini() { + key="$1" value="$2" section="$3" + file="/etc/radicale/config" + sed -ri "/ *\[$section\]/,/^ *\[[^]]+\]/{/^\s*$key[[:space:]=]/d};/ *\[$section\]/a $key = $value" "$file" +} + +# comments say default is 0.0.0.0:5232 +setini hosts 10.8.0.4:5232 server +sgo radicale + +# davdroid from f-droid. username ian, +# url https://cal.iankelling.org +# username ian +# I disabled power management feature, it's got 240 min sync interval, +# so it shouldn't be bad. +# + +# when setting up davdroid, switch to groups are per-contact categories, +# per https://davdroid.bitfire.at/configuration/radicale/ +# +# set account name as ian@iankelling.org, per help text below the +# field. +# +# After setting up account, I added one address book, named +# ian. calender was already created, named ian. checked boxes under +# both. synced. +# +# ignorable background info: +# +# When debugging, tailed /var/log/radicale/radicale.log and nginx log, +# both show the requests happening. Without creating the address book, +# after creating a contact, a sync would delete it. +# +# Address books correspond to .props files in the radicale dir. +# +# Some background is here, +# https://davdroid.bitfire.at/faq/entry/cant-manage-groups-on-device/ +# which shows separate vcard option is from rfc 6350, the other is 2426, +# radicale page says it implements the former not the latter, +# which conflicts with the documentation of which to select, but whatever. +# http://radicale.org/technical_choices/ +# https://davdroid.bitfire.at/faq/entry/cant-manage-groups-on-device/ +# +# Note, url above says only cayanogenmod 13+ and omnirom can manage groups. diff --git a/rootsshsync b/rootsshsync new file mode 100755 index 0000000..9d9140a --- /dev/null +++ b/rootsshsync @@ -0,0 +1,32 @@ +#!/bin/bash +# Copyright (C) 2016 Ian Kelling + +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at + +# http://www.apache.org/licenses/LICENSE-2.0 + +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +set -eE -o pipefail +trap 'echo "$0:$LINENO:error: \"$BASH_COMMAND\" returned $?" >&2' ERR + +[[ $EUID == 0 ]] || exec sudo "$BASH_SOURCE" "$@" + +if test -e /q/root/.ssh; then + dest=/q/root/.ssh + /a/exe/lnf $dest /root +else + dest=/root/.ssh + mkdir -p /root/.ssh + chmod 700 /root/.ssh +fi +# -t times, so it won't rewrite the file every time, +# -L resolve links +rsync -rtL $(eval echo ~${SUDO_USER:-$USER})/.ssh/ $dest +chown -R root:root /root/.ssh diff --git a/spamd-dns-fix b/spamd-dns-fix new file mode 100755 index 0000000..fb042f5 --- /dev/null +++ b/spamd-dns-fix @@ -0,0 +1,16 @@ +#!/bin/bash + +set -eE -o pipefail +trap 'echo "$0:$LINENO:error: \"$BASH_COMMAND\" returned $?" >&2' ERR + +[[ $EUID == 0 ]] + +# to deal with this bug until it\'s fixed +# https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=741521 +# I just happened to notice it in my journal. +str="dns: sendto() to \S\+ failed: Connection refused, failing over" +if journalctl --since=-9m --unit=spamassassin | \ + grep "$str" &>/dev/null; then + echo "dns bug, restarting spamassassin" + systemctl restart spamassassin +fi diff --git a/ssh-emacs-setup b/ssh-emacs-setup new file mode 100755 index 0000000..ee1dd08 --- /dev/null +++ b/ssh-emacs-setup @@ -0,0 +1,38 @@ +#!/bin/bash -l +# Copyright (C) 2016 Ian Kelling + +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at + +# http://www.apache.org/licenses/LICENSE-2.0 + +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +if [[ $EUID != 0 ]]; then + sudo "$0" + exit +fi + +set -eE -o pipefail +trap 'echo "$0:$LINENO:error: \"$BASH_COMMAND\" returned $?"' ERR + +cd $(dirname $0) +teeu /etc/ssh/ssh_config 'SendEnv INSIDE_EMACS BASH_LOGIN_SHELL COLUMNS' +teeu /etc/ssh/sshd_config 'AcceptEnv INSIDE_EMACS BASH_LOGIN_SHELL COLUMNS' +# get rid of useless motd stuff +sed -i --follow-symlinks 's/^\s*PrintLastLog .*/PrintLastLog no/' /etc/ssh/sshd_config +rm -f /etc/update-motd.d/10-help-text /etc/update-motd.d/00-header + + +if isdeb; then + # fyi: debconf-set-selections doesn't like mixing tabs and spaces + echo "debconf debconf/frontend select Readline" | debconf-set-selections + service ssh reload +else + systemctl reload sshd +fi diff --git a/vpn-mail-forward b/vpn-mail-forward new file mode 100755 index 0000000..76da424 --- /dev/null +++ b/vpn-mail-forward @@ -0,0 +1,25 @@ +#!/bin/bash + +set -eE -o pipefail +trap 'echo "$0:$LINENO:error: \"$BASH_COMMAND\" returned $?" >&2' ERR + +do-forward() { + cmd=$1; shift + for port; do + /sbin/iptables -t nat $cmd PREROUTING -i eth0 -p tcp -m tcp --dport $port -j DNAT --to-destination 10.8.0.4:$port + done +} + +ports=(25 143) +case $1 in + start) + do-forward -A ${ports[@]} + ;; + stop) + do-forward -D ${ports[@]} + ;; + *) + echo "$0: error: expected 1 argument of start or stop" + exit 1 + ;; +esac