== Introduction ==
-'''tldr''': For Gnu/Linux (with a bit of Debian bias), a more concise, holistic and automated install than the official Mediawiki docs. Do some initial configuration then download this page and run it, or execute it as you read the well written documentation.
+'''tldr''': For GNU/Linux (with a bit of Debian bias), a more concise, holistic and automated install than the official Mediawiki docs. Do some initial configuration then download this page and run it, or execute it as you read.
''' Goals / Why use this guide? '''
* Explicit automation support wherever practical
* Used to setup this site (style optional)
* Contributions welcome and will be updated/tested on this very site!
-* Support for multiple linux distros
+* Support for multiple gnu/linux distros
* Holistic scope (backups, server setup), but sections stand on their own
* Explicit support for production & local testing instance. Additions for production like https and web analytics.
* Edits to this page are closely monitored by the original author.
'''Assumptions'''
-* Self hosting, single Linux system with Bash shell
+* Self hosting, single GNU/Linux system with Bash shell
* Root shell is assumed throughout
* Code blocks are [https://en.wikipedia.org/wiki/Idempotent idempotent]
'''Email Setup'''
-Setting up email can be an involved process. Mediawiki is perfectly happy to disable email with 1 setting (no password reminders or notifications), but it is a nice feature to have. You could run your own mail server (on the mediawiki server, or elsewhere), or use one of many services which sends mail for very cheap, or free within limits (popular examples [http://www.mailgun.com/ mailgun], [https://mandrillapp.com/ mandrill], [http://www.mailjet.com/pricing mailjet], [https://aws.amazon.com/ses/ aws]), or connect to a full featured send/receive mail provider like [https://fastmail.com fastmail] (this server is setup that way). How we did that is for a future wiki page.
+Setting up email can be an involved process, and this guide assumes that a some program (usually postfix or exim) is implementing a functional sendmail interface. Mediawiki uses email with to send password reminders or notifications, and this guide includes cronjobs for updating mediawiki and doing backups which will send mail in the case of an error. Email is also the recommended way to get notifications of package updates which require manual steps such as restarting of services.
If you are not setting up your server to send mail with a program that uses the default sendmail interface, see these pages when you are configuring mediawiki: [[mediawikiwiki:Manual:$wgEnableEmail|Manual:$wgEnableEmail]], [https://www.mediawiki.org/wiki/Configuration_settings#Email_settings Manual:Email_settings], [[mediawikiwiki:Manual:$wgSMTP|Manual:$wgSMTP]]
# root password for the mysql database
export dbpass=REPLACE_ME
-# git branch for mediawiki + extensions.
-# This guide has only been tested with 1_27.
-# branch names: https://git.wikimedia.org/branches/mediawiki%2Fcore.git
-export mw_branch=REL1_27
+export mwdomain=REPLACE_ME # domain name. for this site, it's ofswiki.org
-# customize these questions to something your contributors would know,
-# and at least doesn't have the answer directly in the question
+# customize these questions. Try not to have the answer be a word in the question.
captchaArray() {
if ! grep -Fx '$localSettingsQuestyQuestions = array (' $mwc; then
tee -a $mwc <<'EOF'
$localSettingsQuestyQuestions = array (
"What is the name of the wiki software this site (and wikipedia) uses?" => "Mediawiki",
- "What does f in ofswiki.org stand for?" => "Free"
+ "REPLACE_ME with a question" => "REPLACE_ME with an answer"
);
EOF
fi
}
+# The rest of this section will work fine with no changes.
+
+# git branch for mediawiki + extensions.
+# This guide has only been tested with 1_27.
+# branch names: https://git.wikimedia.org/branches/mediawiki%2Fcore.git
+export mw_branch=REL1_27
+
# As set by gui installer when choosing cc by sa.
export mw_RightsUrl='https://creativecommons.org/licenses/by-sa/4.0/'
export mw_RightsText='Creative Commons Attribution-ShareAlike'
export mw_RightsIcon='$wgScriptPath/resources/assets/licenses/cc-by-sa.png'
-
-export mwdomain=REPLACE_ME # domain name. for this site, it's ofswiki.org
# Alphanumeric site name for pywikibot.
# Here we use the domain minus the dots, which should work fine without changing.
export mwfamily=${mwdomain//./}
mw-clone https://gerrit.wikimedia.org/r/p/mediawiki/extensions/$ext
if [[ -e $mw/ext/$ext/extension.json ]]; then
# new style extension. remove old style declaration
- sed -i '#^require_once( "\\\$IP/extensions/\$ext/\$ext\.php" );#d' $mwc
+ sed -i --follow-symlinks '#^require_once( "\\\$IP/extensions/\$ext/\$ext\.php" );#d' $mwc
teeu $mwc <<EOF
wfLoadExtension( '$ext' );
EOF
mw-skin() {
local skin=$1
mw-clone https://gerrit.wikimedia.org/r/p/mediawiki/skins/$skin
- sed -i '/^wfLoadSkin/d' $mwc
- sed -i '/^\$wgDefaultSkin/d' $mwc
+ sed -i --follow-symlinks '/^wfLoadSkin/d' $mwc
+ sed -i --follow-symlinks '/^\$wgDefaultSkin/d' $mwc
teeu $mwc <<EOF
\$wgDefaultSkin = "${skin,,*}";
wfLoadSkin( '$skin' );
# main reference:
# https://www.mediawiki.org/wiki/Manual:Running_MediaWiki_on_Ubuntu
apt-get update
- apt-get install -y imagemagick php-mbstring
+ apt-get install -y imagemagick
if apt-get install -s mediawiki &>/dev/null; then
# in debian wheezy time-frame distros, mediawiki was packaged.
- apt-get -y install php-apc mediawiki
+ apt-get -y install php-apc mediawiki php-mbstring
else
# https://www.mediawiki.org/wiki/Manual:Installation_requirements
if apt-get install -s php7.0 &>/dev/null; then
# note, 7.0 is untested by the editor here, since it's not
# available in debian 8. it's listed as supported
# in the mediawiki page.
- # noninteractive to avoid mysql password prompt
+ # noninteractive to avoid mysql password prompt.
DEBIAN_FRONTEND=noninteractive apt-get install -y apache2 mysql-server \
php7.0 php7.0-mysql libapache2-mod-php7.0 php7.0-xml \
- php7.0-apcu
+ php7.0-apcu php7.0-mbstring
else
+ # note: mbstring is recommended, but it's not available for php5 in
+ # debian jessie.
DEBIAN_FRONTEND=noninteractive apt-get install -y apache2 mysql-server \
php5 php5-mysql libapache2-mod-php5 php5-apcu
fi
l=$mw/../../logs
mkdir -p $l
-acme-tiny-wrapper/acme-tiny-wrapper $mwdomain
+acme-tiny-wrapper/acme-tiny-wrapper -t $mwdomain
git clone $git_site/basic-https-conf
{ cat <<EOF
# Increase from default of 2M to 100M.
# This will at least allow high res pics etc.
php_ini=$(php -r 'echo(php_ini_loaded_file());')
-sed -i 's/^\(upload_max_filesize\|post_max_size\)\b.*/\1 = 100M/' $php_ini
+sed -i --follow-symlinks 's/^\(upload_max_filesize\|post_max_size\)\b.*/\1 = 100M/' $php_ini
if isdeb; then
service apache2 restart
else
# Make the toolbox go into the drop down.
cd $mw/skins/Vector
if ! git remote show ian-kelling &>/dev/null; then
- git remote add ian-kelling https://iankelling.org/git/Vector
+ git remote add ian-kelling https://iankelling.org/git/forks/Vector
fi
git fetch ian-kelling
git checkout ian-kelling/REL1_27-toolbox-in-dropdown
# todo, php5-curl equivalent on fedora
yum -y install texlive-cjk ghostscript ImageMagick texlive ocaml
fi
-dir=$(dirname $(php -r 'echo(php_ini_loaded_file());'))/../apache2/conf.d
-ln -sf ../../mods-available/curl.ini $dir
service apache2 restart
cd $mw/extensions/Math/math; make # makes texvc
Enable account creation that we initially disabled.
<source lang="bash">
-sed -i "/\\\$wgGroupPermissions\\['\\*'\\]\\['createaccount'\\] = false;/d" $mwc
+sed -i --follow-symlinks "/\\\$wgGroupPermissions\\['\\*'\\]\\['createaccount'\\] = false;/d" $mwc
</source>
== Additional Configuration with Pywikibot ==
""")
EOF
-python pwb.py ${mwfamily}_setup
+# this can spam a warning, so uniq it
+python pwb.py ${mwfamily}_setup |& uniq
</source>
last_error=0
trap 'last_error=$?' ERR
source ~/mw_vars
+# No strict because the host is likely not named the same as
+# the domain.
+ssh="ssh -oStrictHostKeyChecking=no"
logfile=/var/log/${mwdomain}_backup.log
{
echo "#### starting backup at $(date) ####"
-ssh root@$mwdomain <<ENDSSH
+$ssh root@$mwdomain <<ENDSSH
set -x
tee -a $mwc<<'EOF'
\$wgReadOnly = 'Dumping Database, Access will be restored shortly';
mysqldump -p$dbpass --default-character-set=binary my_wiki > ~/wiki_backups/wiki_db_backup
sed -i '\$ d' $mwc # delete read only setting
ENDSSH
+# add no strict option to the defaults
+
+rdiff() { rdiff-backup --remote-schema "$ssh -C %s rdiff-backup --server" "$@"; }
set -x
-rdiff-backup root@$mwdomain::/root/wiki_backups ~/backup/${mwdomain}_wiki_db_backup
-rdiff-backup root@$mwdomain::$mw ~/backup/${mwdomain}_wiki_file_backup
+rdiff root@$mwdomain::/root/wiki_backups ~/backup/${mwdomain}_wiki_db_backup
+rdiff root@$mwdomain::$mw ~/backup/${mwdomain}_wiki_file_backup
set +x
echo "=== ending backup at $(date) ===="
} &>>$logfile
'''Optional & requires additional steps'''
<source lang="bash" type="example">
-sed -i "/^root:/d" /etc/aliases
+sed -i --follow-symlinks "/^root:/d" /etc/aliases
echo "root: EXAMPLE_ONLY_REPLACE_ME@gmail.com" >> /etc/aliases
newaliases
</source>
'''Whenever you implement a backup system, you should test that restoring the backup works.'''
-You ''should'' be able to restore your wiki to a new machine by repeating all install steps, then restoring the database and the images directory. However, we backup the entire Mediawiki directory in case you forget to record a step, or some misbehaving code stores some state in a file. Since most people don't record the steps they took to setup Mediawiki, this is also the officially recommended method. Here we restore only the database and images folder, which should help identify any of those aforementioned issues. See [[mediawikiwiki:Manual:Restoring a wiki from backup]] if you run into any problems.
+You ''should'' be able to restore your wiki to a new machine by repeating all install steps, then restoring the database and the images directory. I've done this many times. However, we backup the entire Mediawiki directory in case you forget to record a step or some corner case happens. Since most people don't record the steps they took to setup Mediawiki, this is also the officially recommended method. In the code below we restore only the database and images folder from the full backup. You can try this after setting up a wiki from scratch. If it doesn't work, you know your fresh setup is not replicating your backed up wiki correctly. In that case, you can fall back to doing a full restore by copying the full directory instead of just the images. See [[mediawikiwiki:Manual:Restoring a wiki from backup]] if you run into any problems.
To test a backup restore:
# Do a backup of your wiki with some content in it, as described in the previous section
<source lang="bash" type="example">
#!/bin/bash
source ~/mw_vars
-HOSTNAME=REPLACE_ME source ~/mw_vars
-rdiff-backup -r now ~/backup/${mwdomain}_wiki_file_backup /tmp/wiki_file_restore
-scp -r /tmp/wiki_file_restore/images root@$mwdomain:$mw/images
-rdiff-backup -r now ~/backup/${mwdomain}_wiki_db_backup /tmp/wiki_db_restore
-scp -r /tmp/wiki_db_restore root@$mwdomain:/tmp
-ssh root@$mwdomain <<EOF
-mysql -u root -p$dbpass my_wiki < /tmp/wiki_db_restore/wiki_db_dump
+restore="rdiff-backup --force -r now"
+$restore ~/backup/${mwdomain}_wiki_file_backup /tmp/wiki_file_restore
+$restore ~/backup/${mwdomain}_wiki_db_backup /tmp/wiki_db_restore
+o=-oStrictHostKeyChecking=no
+scp $o -r /tmp/wiki_file_restore/images/* root@$mwdomain:$mw/images
+scp $o -r /tmp/wiki_db_restore root@$mwdomain:/tmp
+ssh $o root@$mwdomain <<EOF
+set -e
+chmod -R g+w $mw/images
+chgrp -R www-data $mw/images
+mysql -u root -p$dbpass my_wiki < /tmp/wiki_db_restore/wiki_db_backup
php $mw/maintenance/update.php
EOF
-
</source>
Then browse to your wiki and see if everything appears to work.
#!/bin/bash
source ~/mw_vars
cd $mw
-git fetch --all
-git checkout origin/$mw_branch
-git rebase ian/REL1_23-toolbox-in-dropdown
-cd extensions
-for x in *; do
- if [[ -d $x ]]; then
- cd $x
- git fetch --all
- git checkout origin/$mw_branch || git checkout -qf origin/master
- cd ..
+for dir in extensions/* skins/* $mw; do
+ [[ -d $dir ]] || continue
+ cd $dir
+ branch=$(git describe --all)
+ branch=${branch#remotes/}
+ git fetch --all -q
+ new_head=$(git rev-parse $branch)
+ log=$(git log HEAD..$new_head)
+ if [[ $log ]]; then
+ pwd
+ echo "$log"
fi
+ git checkout -q $new_head
+ cd $mw
done
php $mw/maintenance/update.php -q
EOF
-chmod +x $s
+
</source>
== Upgrading Major Versions ==
Overall the documentation is good, but like wikipedia, it depends.
-The closer a topic is to core functionality and commonly used features, the better the documentation is likely to be. My guess is that Wikimedia Foundation (WMF) has a competing priority of being a good upstream to mediawiki users and being good for their own sites. That, plus the multitude of unconnected extension developers, and official documentation is sometimes neglected in favor of bug reports, readme files, comments, code, and unpublished knowledge. User's edits vary in quality, and often aren't reviewed by anyone. If you run into an issue, try viewing/diffing the most recent version of a page by the last few editors.
+The closer a topic is to core functionality and commonly used features, the better the documentation is likely to be. Wikimedia Foundation (WMF) has a competing priority of being a good upstream to mediawiki users and being good for their own sites. That, plus the multitude of unconnected extension developers, and official documentation is sometimes neglected in favor of bug reports, readme files, comments, code, and unpublished knowledge. User's documentation edits vary in quality, and often aren't reviewed by anyone. If you run into an issue, try viewing/diffing the most recent version of a page by the last few editors.
One issue is that mediawiki.org needs a lot of organizing, deleting, and verifying of material, and that is relatively unpopular, tedious, and sometimes difficult work. The discussion pages of mediawiki.org are a wasteland of unanswered questions and outdated conversations, which is [https://www.mediawiki.org/wiki/Help:Talk_pages poor form] for a wiki. However, if you communicate well, you can get great help from their [https://www.mediawiki.org/wiki/Communication support forum, irc, and mailing list].
'''Bash here documents, EOF vs 'EOF' '''
-Here documents are used throughout this page, some people may not be aware of a small but important syntax. When the delimiter is quoted, as in <<'EOF', then the contents of the here document are exactly verbatim. Otherwise $ and ` are expanded as in bash, and must be quoted by \, which itself must then also be quoted to be used literally.
+Here documents are used throughout this page, some people may not be aware of a small but important syntax. When the delimiter is quoted, as in <<'EOF', then the contents of the here document are exactly verbatim. Otherwise $ and ` are expanded as in bash, and must be escaped by prefixing them with \, which itself must then also be escaped to be used literally.
''' Mediawiki automation tools survey 7/2014 '''
iankelling.org / git / mediawiki-setup / blobdiff