3 '''tldr''': Automated setup on gnu/linux after you set options in [[#Dependent Variables and Functions]], except email. Any code which is not ready to be run has bold text saying so immediately before it.
5 ''' Goals / Why use this guide? '''
7 * Good recommendations. Official docs mostly avoid recommendations among a myriad of possibilities
8 * Closely references & supplements official documentation
9 * Explicit automation support wherever practical
10 * Used to setup this site (style optional)
11 * Contributions welcome. This site will be updated/tested with changes to this document!
12 * Support for multiple linux distros
13 * Holistic scope (backups, server setup), but sections stand on their own
14 * Explicit support for production & local testing instance. Additions for production like https and web analytics.
15 * Edits to this page are closely monitored by the original author.
19 * Self hosting, single Linux system with Bash shell
20 * Root shell is assumed throughout
21 * Code blocks are [https://en.wikipedia.org/wiki/Idempotent idempotent]
25 Very minor adjustments needed for other distros. Help expand this list.
29 Pre 5/2016 revisions ran Mediawiki 1.23, tested on Fedora 20 and Ubuntu 14.04.
31 == Production Server Prerequisites ==
33 '''Getting a Server & a Domain'''
35 The most common route and the one taken by this site is buying a domain name from a site like namecheap, and a cheap vps from companies like linode or digital ocean. They have good getting started guides which mostly apply beyond their own sites.
39 Setting up email can be an involved process. Mediawiki is perfectly happy to disable email with 1 setting (no password reminders or notifications), but it is a nice feature to have. You could run your own mail server (on the mediawiki server, or elsewhere), or use one of many services which sends mail for very cheap, or free within limits (popular examples [http://www.mailgun.com/ mailgun], [https://mandrillapp.com/ mandrill], [http://www.mailjet.com/pricing mailjet], [https://aws.amazon.com/ses/ aws]), or connect to a full featured send/receive mail provider like [https://fastmail.com fastmail] (this server is setup that way). How we did that is for a future wiki page.
41 If you are not setting up your server to send mail with a program that uses the default sendmail interface, see these pages when you are configuring mediawiki: [[mediawikiwiki:Manual:$wgEnableEmail|Manual:$wgEnableEmail]], [https://www.mediawiki.org/wiki/Configuration_settings#Email_settings Manual:Email_settings], [[mediawikiwiki:Manual:$wgSMTP|Manual:$wgSMTP]]
45 There is a balance between effective anti-spam measures and blocking/annoying contributors. Mediawiki documentation on how to combat spam, is not very good, but it has improved over time: [https://www.mediawiki.org/wiki/Manual:Combating_spam manual: Combating Spam]. It's possible for a spammer to quickly make thousands of edits, and there is no good documentation on purging lots of spam, so you should have a good strategy up front. My current strategy is 3 fold, and is limited to small/medium wiki's:
47 * Find new spam quickly, revert it & ban the user.
48 ** Watch, and get notified of changes on all primary content pages: Special:Preferences, Bottom of the page, set an email address, then turn on "Email me also for minor edits of pages and files."
49 ** Use a rss/atom feed reader, and subscribe to recent changes across the wiki. Newer browsers have an rss feed subscribe button, you can click after going to Special:RecentChanges. If that is not available, you can construct the proper url based on [https://meta.wikimedia.org/wiki/Help:Recent_changes#Web_feed these instructions].
50 * Require registration to edit, and a custom captcha question on registration.
51 * Install all non-user inhibiting anti-spam extensions / settings that take a reasonable amount of time to figure out.
53 == Download all ready to use source code blocks into a single script ==
55 Source block that are not ready to use have a bold warning just before them (and a tag on the block) and are skipped.
56 ''' Requires customization: outputs to stdout'''
57 <source lang="bash" type="example">
58 start=' *<source lang="bash"> *'
60 ruby <<'EOF' | sed -rn "/^$start$/,/^$end$/{s/^$start|$end$/# \0/;p}"
62 puts JSON.parse(`curl 'https://ofswiki.org/w/api.php?\
63 action=query&titles=Mediawiki_Setup_Guide&prop=revisions&rvprop=content&\
64 format=json'`.chomp)['query']['pages'].values[0]['revisions'][0]['*']
68 == Dependent Variables and Functions ==
71 # Save the code in this section to a file (~/mw_vars is suggested)
72 # Source it at the beginning of scripts containing later commands
73 # Source it from your .bashrc file while you are setting up Mediawiki
75 '''Requires customization:'''
76 <source lang="bash" type="example">
77 # Replease REPLACE_ME as appropriate
79 export mwdescription="Opinionated Free Software Wiki"
81 # username/pass of the first wiki admin user
82 export wikiuser="Ian Kelling"
83 export wikipass=REPLACE_ME
85 # root password for the mysql database
86 export dbpass=REPLACE_ME
88 # password for piwik web analytics admin
89 export piwik_pass=REPLACE_ME
91 # git branch for mediawiki + extensions
92 # branch names: https://git.wikimedia.org/branches/mediawiki%2Fcore.git
93 export mw_branch=REL1_27
95 # customize these questions to something your contributors would know,
96 # is not super common / scriptable, and doesn't have the answer in the question
98 if ! grep -Fx '$localSettingsQuestyQuestions = array (' $mwc; then
100 $localSettingsQuestyQuestions = array (
101 "What is the name of the wiki software this site (and wikipedia) uses?" => "Mediawiki",
102 "What does f in ofswiki.org stand for?" => "Free"
107 # As set by gui installer when choosing cc by sa.
108 export mw_RightsUrl="https://creativecommons.org/licenses/by-sa/4.0/";
109 export mw_RightsText="Creative Commons Attribution-ShareAlike";
110 export mw_RightsIcon="$wgResourceBasePath/resources/assets/licenses/cc-by-sa.png";
113 # domain name and install paths. These are examples, change as needed.
114 export mwfamily=ofswiki # short alphanumeric name for pywikibot
115 export mwdomain=ofswiki.org
116 export mw=/var/www/$mwdomain/html/w
119 # wiki sender address / wiki & wiki server contact email.
120 # see email section for more info on email
121 export mw_email="admin@$mwdomain"
124 Here we define some small useful bash functions. This can be part of the same file.
127 # for convenience, Mediawiki config file
128 mwc="$mw/LocalSettings.php"
130 # identify if this is a debian based distro
131 isdeb() { command -v apt &>/dev/null; }
132 # tee unique. append each stdin line if it does not exist in the file
136 for line in "${MAPFILE[@]}"; do
137 grep -xFq "$line" "$1" &>/dev/null || tee -a "$1" <<<"$line"
141 # get and reset an extension/skin repository, and enable it
142 mw-ext () { mw-extra extensions $@; }
143 mw-skin() { mw-extra skins $@; }
146 local type=$1 # extension or skin
150 local original_pwd="$PWD"
151 # it's ok that this fails if we already have it
152 url=https://git.wikimedia.org/git/mediawiki
153 target=$mw/$type/$ext
154 if [[ ! -e $target/.git ]]; then
155 git clone $url/$type/$ext.git $target
157 if ! cd $target; then
158 echo "mw-ext error: failed cd $mw/extensions/$ext";
162 git checkout -qf origin/$mw_branch || git checkout -qf origin/master
167 if [[ -e $target/extension.json ]]; then
168 # new style extension. remove old style declaration
169 sed -i '#^require_once( "\\\$IP/extensions/\$ext/\$ext\.php" );#d' $mwc
171 wfLoadExtension( '$ext' );
175 require_once( "\$IP/extensions/$ext/$ext.php" );
180 sed -i '/^wfLoadSkin/d' $mwc
181 sed -i '/^\$wgDefaultSkin/d' $mwc
183 \$wgDefaultSkin = "${ext,,*}";
184 wfLoadSkin( '$ext' );
189 # --quick is quicker than default flags,
190 # but still add a sleep to make sure everything works right
191 sudo -u $apache_user php $mw/maintenance/update.php -q --quick; sleep 1
194 if command -v apt &>/dev/null; then
202 == Install Mediawiki Dependencies ==
204 The best way to get core dependencies is to install the mediawiki package itself. Nothing about it will get in the way of using a version from upstream.
206 [[mediawikiwiki:Main Page|Mediawiki Main Page]]: the beginning of the official docs.
208 [[mediawikiwiki:Manual:Installation_requirements|Manual:Installation_requirements]]: Overview of installation requirements.
210 Note, this guide needs a little adjustment before it will work with php7.0: make sure settings are still valid, update ini path.
215 # From here on out, exit if a command fails.
216 # This will prevent us from not noticing an important failure.
217 # We recommend setting this for the entire installation session.
218 # If you are running commands interactively, it might be best to
219 # put it in your ~/.bashrc temporarily.
221 trap 'echo "$0:$LINENO:error: \"$BASH_COMMAND\" returned $?" >&2' ERR
226 # https://www.mediawiki.org/wiki/Manual:Running_MediaWiki_on_Ubuntu
228 apt-get install -y ImageMagick
229 if apt-get install -s mediawiki &>/dev/null; then
230 # in debian wheezy time-frame distros, mediawiki was packaged.
231 apt-get -y install php-apc mediawiki
233 # https://www.mediawiki.org/wiki/Manual:Installation_requirements
234 if apt-get install -s php7.0 &>/dev/null; then
235 # note, 7.0 is untested by the editor here, since it's not
236 # available in debian 8. it's listed as supported
237 # in the mediawiki page.
238 # noninteractive to avoid mysql password prompt
239 DEBIAN_FRONTEND=noninteractive apt-get install -y apache2 mysql-server \
240 php7.0 php7.0-mysql libapache2-mod-php7.0 php7.0-xml \
243 DEBIAN_FRONTEND=noninteractive apt-get install -y apache2 mysql-server \
244 php5 php5-mysql libapache2-mod-php5 php5-apcu
247 service apache2 restart
250 # fedora deps are missing a database, so some is translated from debian packages
251 yum -y install mediawiki ImageMagick php-mysqlnd php-pecl-apcu mariadb-server
253 systemctl restart mariadb.service
254 systemctl enable mariadb.service
255 systemctl enable httpd.service
256 systemctl restart httpd.service
260 # slightly different depending on if we already set the root pass
261 if echo exit|mysql -u root -p"$dbpass"; then
262 # answer interactive prompts:
263 # mysql root pass, change pass? no, remove anon users? (default, yes)
264 # disallow remote root (default, yes), reload? (default, yes)
265 echo -e "$dbpass\nn\n\n\n\n" | mysql_secure_installation
267 # I had 1 less newline at the start when doing ubuntu 14.04,
268 # compared to debian 8, so can't say this is especially portable.
269 # It won't hurt if it fails.
270 echo -e "\n\n$dbpass\n$dbpass\n\n\n\n\n" | mysql_secure_installation
275 '''Skippable notes'''
278 php[5]-mysqlnd is a faster mysql driver package, but the default in debian php-mysql, appparently because some non-mediawiki packages are not compatible with it. If you run into this issue, simply use the php-mysql package.
281 Additional packages rational
282 * ImageMagick is [https://www.mediawiki.org/wiki/Manual:Image_administration#Image_thumbnailing recommended].
283 * Gui install and [[mediawikiwiki:Manual:Cache]] recomend the apc package.
284 * Clamav for virus scanning of uploads is mentioned in the mediawiki manual. However, wikipedia doesn't seem to do it, so it doesn't seem like it's worth bothering. It also makes uploading a set of images take twice as long on broadband.
286 == Install Mediawiki ==
289 Here, we [[mediawikiwiki:Download_from_Git]], or reset our installation if it is already there, and create the wiki database. [[mediawikiwiki:Manual:Installing_MediaWiki]]
294 # this will just fail if it already exists which is fine
295 if [[ ! -e .git ]]; then
296 git clone https://gerrit.wikimedia.org/r/p/mediawiki/core.git .
298 # to see available branches: https://www.mediawiki.org/wiki/Version_lifecycle
301 git checkout -f origin/$mw_branch
303 # Get the php libraries wmf uses. Based on:
304 # https://www.mediawiki.org/wiki/Download_from_Git#Fetch_external_libraries
305 if [[ ! -e vendor/.git ]]; then
306 git clone https://gerrit.wikimedia.org/r/p/mediawiki/vendor.git
309 git checkout -f origin/$mw_branch
312 # Drop any previous database which may have been installed while testing.
313 # If upgrading, we should have a db backup which will get restored.
314 # https://www.mediawiki.org/wiki/Manual:Upgrading
315 mysql -u root -p$dbpass <<'EOF' ||:
316 drop database my_wiki;
319 php $mw/maintenance/install.php --pass $wikipass --scriptpath /w \
320 --dbuser root --dbpass $dbpass "$mwdescription" "$wikiuser"
322 # lock down the wiki to only the initial owner until anti-spam measures are put in place
323 # limit edits to registered users
324 $wgGroupPermissions['*']['edit'] = false;
325 # don't allow any account creation
326 $wgGroupPermissions['*']['createaccount'] = false;
331 Note: When testing, you may need to clear the apc cache to see changes take effect in the browser. Simplest solution is
332 just restart apache. http://stackoverflow.com/questions/911158/how-to-clear-apc-cache-entries
334 ''' Skippable Notes'''
336 If we wanted to reset our installation, but leave the extension repositories alone, alter the command above to be <code>git clean -fxd</code>
338 '''Rational for choosing git sources'''
340 Upstream vs distro packages. Upstream is responsive, and it's distributed within a single directory, so packaging does not integrate with the distro's filesystem. The only potential value would be less bugs by using stable versions, but we choose not to make that tradeoff.
342 Why use git over zip file releases? Mediawiki supports git usage through release branches which get post-release fixes. This means we can auto-update, get more granular fixes, easier to manage updates, and rollbacks.
344 == Get Piwik Files ==
346 Because it's really fun to look at your traffic stats. This is only needed for a production site. We get these files now so that we can take them into account when configuring apache in the next section.
348 [http://piwik.org/docs/installation-maintenance/ Overall installation guide]
351 tmpdir="$(mktemp -d)"
353 wget http://builds.piwik.org/piwik.zip
354 if isdeb; then apt-get -y install unzip; else yum -y install unzip; fi
356 # gui installer suggested command
358 chown -R www-data:www-data piwik
360 chown -R apache:apache piwik
362 # remove any existing directory
363 rm -rf $mw/../analytics
364 mv piwik $mw/../analytics
370 == Configure Apache ==
372 Note, non-debian based installs: modify instructions below to use /etc/httpd/conf.d/$mwdomain.conf, and don't run a2ensite.
374 I use scripts I maintains separately to setup Let's Encrypt certificates and apache config: (url pending).
376 If you are doing a test setup on your local machine, you can make your domain resolve to your local test installation, then remove it later when you are done. Note, you will need non-local site to get Let's Encrypt certificates, and then transfer them locally, or disable ssl from the apache config (neither is covered here) and replace all instances of https in these instructions with http. Another option is to get a cheap 2 dollar domain for your test site.
378 '''Not for production:'''
379 <source lang="bash" type="example">
380 teeu /etc/hosts<<<"127.0.0.1 $mwdomain"
383 To not use my scripts, and still use Let's Encrypt: follow this doc page: https://letsencrypt.org/getting-started/. It's a little long winded, so I would boil it down to this:
385 '''Optional & requires additional steps:'''
386 <source lang="bash" type="example">
387 git clone https://github.com/certbot/certbot
389 ./certbot-auto apache
390 cd /etc/apache/sites-available
391 mv 000-default-le-ssl.conf $mwdomain.conf
392 rm ../sites-enabled/000-default-le-ssl.conf
393 # edit $mwdomain.conf, so documentroot is /var/www/$mwdomain/html
394 # and ServerName is $mwdomain
395 a2ensite $mwdomain.conf
397 Then, copy the input to apache-site below and insert it into the apache config.
399 Here, we use some scripts automate setting up the Let 's Encrypt cert and
405 git_site=https://iankelling.org/git
406 git clone $git_site/acme-tiny-wrapper
410 acme-tiny-wrapper/acme-tiny-wrapper $mwdomain
412 git clone $git_site/basic-https-conf
414 ServerAdmin $mw_email
416 # make the site's root url go to our main page
417 RewriteRule ^/?wiki(/.*)?\$ %{DOCUMENT_ROOT}/w/index.php [L]
418 # use short urls https://www.mediawiki.org/wiki/Manual:Short_URL
419 RewriteRule ^/*\$ %{DOCUMENT_ROOT}/w/index.php [L]
421 find -L $(readlink -f $mw/..) -name .htaccess \
422 | while read line; do
423 echo -e "<Directory ${line%/.htaccess}>\n $(< $line)\n</Directory>";
425 } | basic-https-conf/apache-site -r ${mw%/*} - $mwdomain
428 Now mediawiki should load in your browser at $mwdomain .
430 Allow proper search bots and internet archiver bots, via [[Mediawiki:Robots.txt]],
431 and install the default skin.
434 dd of=$mw/../robots.txt <<'EOF'
437 User-agent: ia_archiver
443 '''Skippable Notes'''
445 This section assumes we are redirecting www to a url without www.
447 [http://httpd.apache.org/docs/current/howto/htaccess.html Apache recommends] moving .htaccess rules into it's config for performance. So we look for .htaccess files from mediawiki and piwik and copy their contents into this config. In modern apache versions, we would have to explicitly set options like AllowOverride to allow .htaccess files to take effect.
449 == Mediawiki Settings ==
451 Overall reference: [[mediawikiwiki:Manual:Configuration_settings]].
453 Settings which the gui setup prompts for but aren't set by the automated install script.
456 \$wgServer = "https://$mwdomain";
457 \$wgDBserver = "localhost";
458 \$wgRightsUrl = "$mw_RightsUrl";
459 \$wgRightsText = "$mw_RightsText";
460 \$wgRightsIcon = "$mw_RightsIcon";
463 Settings I recommend which are different than the defaults.
466 \$wgPasswordSender = "$mw_email";
467 \$wgEmergencyContact = "$mw_email";
468 \$wgEnotifUserTalk = true; # UPO
469 \$wgEnotifWatchlist = true; # UPO
470 \$wgMainCacheType = CACHE_ACCEL;
471 \$wgEnableUploads = true;
472 \$wgUseInstantCommons = true;
479 # from https://www.mediawiki.org/wiki/Manual:Short_URL
480 $wgArticlePath = "/wiki/$1";
482 # https://www.mediawiki.org/wiki/Manual:Combating_spam
483 # check that url if our precautions don't work
484 # not using nofollow is good practice, as long as we avoid spam.
485 $wgNoFollowLinks = false;
486 # Allow user customization.
487 $wgAllowUserJs = true;
488 $wgAllowUserCss = true;
490 # use imagemagick over GD
491 $wgUseImageMagick = true;
495 # https://www.mediawiki.org/wiki/Manual:Configuring_file_uploads
496 # Increase from default of 2M to 100M.
497 # This will at least allow high res pics etc.
498 php_ini=$(isdeb && echo /etc/php5/apache2/php.ini || echo /etc/php.ini)
499 sed -i 's/^\(upload_max_filesize\|post_max_size\)\b.*/\1 = 100M/' $php_ini
501 service apache2 restart
503 systemctl restart httpd.service
506 # if you were to install as a normal user, you would need this for images
507 # sudo usermod -aG $apache_user $USER
509 # this doesn't propogate right away
510 chgrp -R $apache_user $mw/images
511 chmod -R g+w $mw/images
514 Style settings. Omit to use a different style.
518 #$wgFooterIcons = null;
520 # Make the toolbox go into the drop down.
522 if ! git remote show ian-kelling &>/dev/null; then
523 git remote add ian-kelling https://iankelling.org/git/Vector
525 git fetch ian-kelling
526 git checkout ian-kelling/REL1_27-toolbox-in-dropdown
529 == Install and Configure Mediawiki Extensions ==
531 When installing extensions on a wiki with important content, backup first as a precaution.
533 ''' Extensions with no configuration needed '''
539 | [[mediawikiwiki:Extension:Cite|Extension:Cite]]
540 | Have references in footnotes.
542 | [[mediawikiwiki:Extension:CiteThisPage|Extension:CiteThisPage]]
543 | Ability to generate citations to pages in a variety of styles.
545 | [[mediawikiwiki:Extension:CSS|Extension:CSS]]
546 | Allows CSS stylesheets to be included in specific articles
548 | [[mediawikiwiki:Extension:Echo|Extension:Echo]]
549 | Notification subsystem for usage by other extensions
551 | [[mediawikiwiki:Extension:Gadgets|Extension:Gadgets]]
552 | UI extension system for users
554 | [[mediawikiwiki:Extension:ImageMap|Extension:ImageMap]]
555 | Links for a region of an image
557 | [[mediawikiwiki:Extension:Interwiki|Extension:Interwiki]]
558 | Tool for nice links to other wikis
560 | [[mediawikiwiki:Extension:News|Extension:News]]
561 | Embed or rss recent changes
563 | [[mediawikiwiki:Extension:Nuke|Extension:Nuke]]
564 | Mass delete of pages, in the case of spam
566 | [[mediawikiwiki:Extension:ParserFunctions|Extension:ParserFunctions]]
567 | Useful for templates
569 | [[mediawikiwiki:Extension:Poem|Extension:Poem]]
570 | Useful for formatting things various ways
572 | [[mediawikiwiki:Extension:SyntaxHighlight_GeSHi|Extension:SyntaxHighlight_GeSHi]]
573 | Source code highlighting
575 | [[mediawikiwiki:Extension:Variables|Extension:Variables]]
576 | Define per-page variables
580 mw-ext Cite CiteThisPage CSS Echo Gadgets ImageMap Interwiki News \
581 Nuke ParserFunctions Poem SyntaxHighlight_GeSHi Variables
585 ''' [[mediawikiwiki:Extension:AntiSpoof|Extension:AntiSpoof]]: Disallow usernames with unicode trickery to look like existing names'''
589 # recommended setup script to account for existing users
590 sudo -u $apache_user php $mw/extensions/AntiSpoof/maintenance/batchAntiSpoof.php
594 ''' [[mediawikiwiki:CheckUser|Extension:CheckUser]]: Get ip addresses from inside mediawiki so you can ban users'''
596 Requires special install steps or we can get into a bad state. Add a sleep like the default of update.php to avoid errors.
599 sudo -u $apache_user php $mw/extensions/CheckUser/install.php; sleep 1
603 '''[[mediawikiwiki:Extension:Wikidiff2|Extension:Wikidiff2]]: Faster and international character supported page diffs'''
605 I used packaged version since this is a c++ and probably not very tied to the Mediawiki version. This isn't packaged in fedora, haven't gotten around to testing and adding the code to compile it for fedora.
608 apt-get -y install php-wikidiff2
610 $wgExternalDiffEngine = 'wikidiff2';
612 ln -sf ../../mods-available/wikidiff2.ini /etc/php5/apache2/conf.d
613 service apache2 restart
618 ''' [[mediawikiwiki:Extension:Math|Extension:Math]] Display equations'''
622 # php5-curl according to Math readme
625 if ! apt-get -s install $curl_pkg &>/dev/null; then
628 apt-get -y install latex-cjk-all texlive-latex-extra texlive-latex-base \
629 ghostscript imagemagick ocaml $curl_pkg make
631 # todo, php5-curl equivalent on fedora
632 yum -y install texlive-cjk ghostscript ImageMagick texlive ocaml
634 ln -sf ../../mods-available/curl.ini /etc/php5/apache2/conf.d
635 service apache2 restart
637 cd $mw/extensions/Math/math; make # makes texvc
638 cd $mw/extensions/Math/texvccheck; make
641 # Enable MathJax as rendering option
642 $wgUseMathJax = true;
643 # Enable LaTeXML as rendering option
644 $wgMathValidModes[] = 'latexml';
645 # Set LaTeXML as default rendering option, because it is nicest
646 $wgDefaultUserOptions['math'] = 'latexml';
650 '''Skippable notes'''
652 There is no current list of package depencies so I took dependencies from mediawiki-math package in Debian 7. Fedora didn't have a mediawik math package, so I just translated from debian. Ocaml is for math png rendering, as backup option to the nicer looking LatexML and MathJax. Debian has texvc package, but it didn't work right for me, plus it required additional configuration in mediawiki settings.
655 ''' [[mediawikiwiki:Extension:SpamBlacklist|Extension:SpamBlacklist]]: Import/create IP blacklists, mainly for spam'''
659 if ! grep -F '$wgSpamBlacklistFiles = array(' $mwc &>/dev/null; then
661 $wgEnableDnsBlacklist = true;
662 $wgDnsBlacklistUrls = array( 'xbl.spamhaus.org', 'dnsbl.tornevall.org' );
664 ini_set( 'pcre.backtrack_limit', '10M' );
665 $wgSpamBlacklistFiles = array(
666 "[[m:Spam blacklist]]",
667 "http://en.wikipedia.org/wiki/MediaWiki:Spam-blacklist"
673 ''' [[mediawikiwiki:Extension:TitleBlacklist|Extension:TitleBlacklist]]: Anti-spam '''
676 mw-ext TitleBlacklist
677 if ! grep -F '$wgTitleBlacklistSources = array(' $mwc &>/dev/null; then
679 $wgTitleBlacklistSources = array(
682 'src' => 'MediaWiki:Titleblacklist',
686 'src' => 'http://meta.wikimedia.org/w/index.php?title=Title_blacklist&action=raw',
693 ''' [[mediawikiwiki:Extension:WikiEditor|Extension:WikiEditor]]: Editing box extras and a fast preview tab '''
698 # Enable Wikieditor by default
699 $wgDefaultUserOptions['usebetatoolbar'] = 1;
700 $wgDefaultUserOptions['usebetatoolbar-cgd'] = 1;
702 # Display the Preview and Changes tabs
703 $wgDefaultUserOptions['wikieditor-preview'] = 1;
707 ''' [[mediawikiwiki:CategoryTree|Extension:CategoryTree]]: Enables making nice outlines of pages in a category'''
711 # Mediawiki setting dependency for CategoryTree
716 ''' [[mediawikiwiki:Extension:AbuseFilter|Extension:AbuseFilter]]: Complex abilities to stop abuse '''
718 Used by big wiki sites. As a smaller site, we won't use it much, but it's good to have. It's page suggests a few defaults:
722 $wgGroupPermissions['sysop']['abusefilter-modify'] = true;
723 $wgGroupPermissions['*']['abusefilter-log-detail'] = true;
724 $wgGroupPermissions['*']['abusefilter-view'] = true;
725 $wgGroupPermissions['*']['abusefilter-log'] = true;
726 $wgGroupPermissions['sysop']['abusefilter-private'] = true;
727 $wgGroupPermissions['sysop']['abusefilter-modify-restricted'] = true;
728 $wgGroupPermissions['sysop']['abusefilter-revert'] = true;
732 '''[[mediawikiwiki:Extension:ConfirmEdit|Extension:ConfirmEdit]]: Custom Captcha'''
734 Uses captchaArray defined in mw_vars.
740 wfLoadExtension( 'ConfirmEdit/QuestyCaptcha' );
741 $wgCaptchaClass = 'QuestyCaptcha';
742 # only captcha on registration
743 $wgGroupPermissions['user' ]['skipcaptcha'] = true;
744 $wgGroupPermissions['autoconfirmed']['skipcaptcha'] = true;
746 if ! grep -Fx 'foreach ( $localSettingsQuestyQuestions as $key => $value ) {' $mwc; then
748 foreach ( $localSettingsQuestyQuestions as $key => $value ) {
749 $wgCaptchaQuestions[] = array( 'question' => $key, 'answer' => $value );
755 Enable account creation that we initially disabled.
757 sed -i "/\\\$wgGroupPermissions\\['\\*'\\]\\['createaccount'\\] = false;/d" $mwc
760 == Additional Configuration with Pywikibot ==
762 There are quite a few [[mediawikiwiki:Help:Namespaces|special pages]] which act like variables to configure special wiki content and style. A big part of this wiki's style is configured in this section. We use Pywikibot to automate editing those pages.
765 '''Pywikibot Install'''
767 [[mediawikiwiki:Manual:Pywikibot/Installation|Manual:Pywikibot/Installation]]
771 if [[ ! -e ~/pywikibot/.git ]]; then
772 git clone --recursive \
773 https://gerrit.wikimedia.org/r/pywikibot/core.git ~/pywikibot
782 '''Pywikibot Configuration'''
784 Relevent docs: [[mediawikiwiki:Manual:Pywikibot/Use_on_non-WMF_wikis|Manual:Pywikibot/Use_on_non-WMF_wikis]], [[mediawikiwiki:Manual:Pywikibot/Quick_Start_Guide|Manual:Pywikibot/Quick_Start_Guide]]
789 dd of=user-config.py <<EOF
791 usernames["$mwfamily"]['en'] = u'$wikiuser'
793 console_encoding = 'utf-8'
794 password_file = "secretsfile"
797 dd of=secretsfile <<EOF
798 ("$wikiuser", "$wikipass")
801 # it won't overrwrite an existing file. Remove if if one exists
802 rm -f pywikibot/families/${mwfamily}_family.py
804 apt-get install -y python-requests
806 yum -y install python-requests
809 python generate_family_file.py https://$mwdomain/wiki/Main_Page "$mwfamily"
811 # Note, this needed only for ssl site
812 tee -a pywikibot/families/${mwfamily}_family.py<<'EOF'
813 def protocol(self, code):
819 '''Pywikibot Script'''
821 This will take a full minute or so because the bot waits a few seconds between edits. Useful doc: [[mediawikiwiki:Pywikipediabot/Create_your_own_script]].
826 dd of=scripts/${mwfamily}_setup.py<<EOF
830 site = pywikibot.Site()
832 page = pywikibot.Page(site, p)
834 #force is for some anti-bot thing, not necessary in my testing, but might as well include it
835 page.save(force=True)
837 # Small/medium noncommercial wiki should be fine with no privacy policy
838 # based on https://www.mediawiki.org/wiki/Manual:Footer
839 x("MediaWiki:Privacy")
841 # licenses for uploads. Modified from the mediawiki's wiki
842 x("MediaWiki:Licenses", u"""* Same as this wiki's text (preferred)
843 ** CC BY-SA or GFDL| Creative Commons Attribution ShareAlike or GNU Free Documentation License
845 ** Unknown_copyright|I don't know exactly
846 ** PD|PD: public domain
847 ** CC BY|Creative Commons Attribution
848 ** CC BY-SA|Creative Commons Attribution ShareAlike
849 ** GFDL|GFDL: GNU Free Documentation License
850 ** GPL|GPL: GNU General Public License
851 ** LGPL|LGPL: GNU Lesser General Public License""")
852 x("MediaWiki:Copyright", '$mw_license')
853 x("MediaWiki:Mainpage-description", "$mwdescription")
857 # The rest of the settings are for the site style
859 # Remove various clutter
860 x("MediaWiki:Lastmodifiedat")
861 x("MediaWiki:Disclaimers")
862 x("MediaWiki:Viewcount")
863 x("MediaWiki:Aboutsite")
864 # remove these lines from sidebar
865 # ** recentchanges-url|recentchanges
866 # ** randompage-url|randompage
868 x("MediaWiki:Sidebar", """* navigation
869 ** mainpage|mainpage-description
875 # helpfull doc: https://www.mediawiki.org/wiki/Manual:Interface/Sidebar
876 x("mediawiki:Common.css", """/* adjust sidebar to just be home link and up top */
877 /* panel width increased to fit full wiki name. */
878 div#mw-panel { top: 10px; padding-top: 0em; width: 20em }
879 div#footer, #mw-head-base, div#content { margin-left: 1em; }
880 #left-navigation { margin-left: 1em; }
883 /* logo, and toolbar hidden */
884 #p-logo, #p-tb.portal {
888 /* make the font size smaller for the misc stuff */
896 div#mw-content-text {
902 python pwb.py ${mwfamily}_setup
906 ''' Skippable Notes '''
908 The docs suggest manually entering the pass with python pwb.py login.py, then it should stay logged in. That didn't work for me, and anyways, we automation, so we use secrets file method.
910 Family name, and all its duplicattions documented as supposed to be $wgSitename, but it works fine using any name.
912 == Automatic Backups ==
914 Here we will have a daily cronjob where a backup host sshs to the mediawiki host, makes a backup then copies it back. Copy ~/mw_vars to the backup host at /root/mw_vars. Setup passwordless ssh from the backup host to the mediawiki host. Then run this code on the backup host. This will make a versioned backup of the wiki to ~/backup.
916 <source lang="bash" type="backup">
917 backup_script=/etc/cron.daily/mediawiki_backup
918 sudo dd of=$backup_script <<'EOFOUTER'
920 # if we get an error, keep going but return it at the end
922 trap 'last_error=$?' ERR
924 logfile=/var/log/${mwdomain}_backup.log
926 echo "#### starting backup at $(date) ####"
927 ssh root@$mwdomain <<ENDSSH
930 \$wgReadOnly = 'Dumping Database, Access will be restored shortly';
932 mkdir -p ~/wiki_backups
933 mysqldump -p$dbpass --default-character-set=binary my_wiki > ~/wiki_backups/wiki_db_backup
934 sed -i '\$ d' $mwc # delete read only setting
937 rdiff-backup root@$mwdomain::/root/wiki_backups ~/backup/${mwdomain}_wiki_db_backup
938 rdiff-backup root@$mwdomain::$mw ~/backup/${mwdomain}_wiki_file_backup
940 echo "=== ending backup at $(date) ===="
942 if [[ $last_error != 0 ]]; then
943 echo "backup for $mwdomain failed. See $logfile"
948 sudo chmod +x $backup_script
951 If you are like most people and don't use the old-school mail spool, setup the backup system to send mail externally. Some ways to do that are on this [http://unix.stackexchange.com/questions/36982/can-i-set-up-system-mail-to-use-an-external-smtp-server stackoverflow answer]. Then make local mail to your user get forwarded to an address you will read:
953 '''Optional & requires additional steps'''
954 <source lang="bash" type="example">
955 sed -i "/^root:/d" /etc/aliases
956 echo "root: EXAMPLE_ONLY_REPLACE_ME@gmail.com" >> /etc/aliases
960 == Restoring Backups ==
962 '''Whenever you implement a backup system, you should test that restoring the backup works.'''
964 You ''should'' be able to restore your wiki to a new machine by repeating all install steps, then restoring the database and the images directory. However, we backup the entire Mediawiki directory in case you forget to record a step, or some misbehaving code stores some state in a file. Since most people don't record the steps they took to setup Mediawiki, this is also the officially recommended method. Here we restore only the database and images folder, which should help identify any of those aforementioned issues. See [[mediawikiwiki:Manual:Restoring a wiki from backup]] if you run into any problems.
966 To test a backup restore:
967 # Do a backup of your wiki with some content in it, as described in the previous section
968 # Move your mediawiki install directory, or setup Mediawiki on a new machine
969 # Re-execute the mediawiki install steps
970 # Change REPLACE_ME in the code below (as in the backup section so you get the right variables),
971 # Execute the code on the backup machine.
973 <source lang="bash" type="example">
976 HOSTNAME=REPLACE_ME source ~/mw_vars
977 rdiff-backup -r now ~/backup/${mwdomain}_wiki_file_backup /tmp/wiki_file_restore
978 scp -r /tmp/wiki_file_restore/images root@$mwdomain:$mw/images
979 rdiff-backup -r now ~/backup/${mwdomain}_wiki_db_backup /tmp/wiki_db_restore
980 scp -r /tmp/wiki_db_restore root@$mwdomain:/tmp
981 ssh root@$mwdomain <<EOF
982 mysql -u root -p$dbpass my_wiki < /tmp/wiki_db_restore/wiki_db_dump
983 php $mw/maintenance/update.php
988 Then browse to your wiki and see if everything appears to work.
992 Subscribe to get release and security announcements [https://lists.wikimedia.org/mailman/listinfo/mediawiki-announce MediaWiki-announce].
994 For updates, we simply git pull all the repos, then run the maintenance script. This should be done after a backup. We recommend automatic updates to get security fixes and since not much is changing on the release branch. In this example, we update at 5 am daily (1 hour after the automatic backup example).
996 Major version upgrades should be done manually, and it is recommended to use a new installation directory and the same procedure as for backup & restore. Official reference: [[mediawikiwiki:Manual:Upgrading|Manual:Upgrading]]
998 Minor updates script:
1000 s=/etc/cron.daily/mediawiki_update
1006 git checkout origin/$mw_branch
1007 git rebase ian/REL1_23-toolbox-in-dropdown
1010 if [[ -d $x ]]; then
1013 git checkout origin/$mw_branch || git checkout -qf origin/master
1017 php $mw/maintenance/update.php -q
1022 == Upgrading Major Versions ==
1024 Reference documentation is at [[mediawikiwiki:Manual:Upgrading]]
1028 # Read the "Upgrade notices for MediaWiki administrators" on the upgrade version and any skipped versions at [[mediawikiwiki:Version_lifecycle]].
1029 # Setup a blank test wiki with the new version.
1030 # Backup the old database, restore it to the new wiki, run php maintenance/update.php.
1031 # If everything looks good, repeat and replace the old wiki with the new one.
1036 <source lang="bash">
1037 # based on recommendation from install page
1038 if ! dpkg -s php5-gd &>/dev/null; then
1039 apt-get install -y php5-gd
1040 service apache2 restart
1043 # docs suggests using separate database user
1044 mysql -u root -p$dbpass <<EOF
1045 GRANT ALL ON piwik.* TO 'piwik'@'localhost' IDENTIFIED BY '$piwik_pass';
1049 php_ini=$(isdeb && echo /etc/php5/apache2/php.ini || echo /etc/php.ini)
1050 # based on the install page. however, ths option is changing with php7.0
1051 opt=always_populate_raw_post_data
1052 sed -ri "/^ *$opt\b/d;/^ *\[PHP\]/a $opt = -1" $php_ini
1053 service apache2 restart
1059 Unfortunately, Piwik does not document how to do a fully automated install, and although it certainly seems possible, my first glance at the config file after doing manual steps showed a lot of noise, so we are sticking with the manual steps for now.
1061 In a browser, navigate to your_domain/analytics. Follow these steps [http://piwik.org/docs/installation/#the-5-minute-piwik-installation the-5-minute-piwik-installation]. Use the user/db/pass from the previous section.
1064 We are going to use an extension to handle the javaScript tracking tag part.
1066 Under settings -> user settings, click the cookie to exclude your current browser.
1069 ''' [[mediawikiwiki:Extension:Piwik Integration|Extension:Piwik Integration]] '''
1071 <source lang="bash">
1072 git clone https://github.com/DaSchTour/piwik-mediawiki-extension.git $mw/extensions/Piwik
1075 \$wgPiwikURL = '$mwdomain/analytics/';
1076 \$wgPiwikIDSite = '1';
1081 '''If this is not the first site you setup'''
1083 You will need to adjust the previous $wgPiwikIDSite variable. The correct value can be found in the Piwik javaScript tracking tag code, in the line that looks like <code>_paq.push(['setSiteId', 2]);</code>
1087 Based on [http://piwik.org/docs/update/ piwik.org/docs/update]. Previously it's been seen to require manual steps, so it is not recommended to put in a cron script.
1089 <source lang="bash" type="example">
1091 piwik_path=$mw/../analytics
1093 wget -q http://builds.piwik.org/piwik.zip
1096 cp $piwik_path/config/config.ini.php piwik/config
1097 cp -rf piwik/* $piwik_path
1098 # prevent making an email out of the standard success response
1099 $piwik_path/console core:update
1103 ''' Skippable Notes '''
1105 Piwik docs say use a subdir or a subdomain. googling finds a page which suggests using a subdomain, for more logical separation https://library.linode.com/web-applications/analytics/piwik/centos-5. However, mediawiki doesn't use the root dir of the web server, so the analytics folder can't mess things up by existing there, and it doesn't have any crazy virtualhost settings which might conflict, so there is no need. Plus we get ssl for free by sharing the same domain.
1107 The docs also say: "When you have finished configuring Piwik, you can set more restrictive permissions (ie. read only) to the “piwik/config/” folder. Piwik will always write its data inside the “piwik/tmp/” folder, so this is the only folder that requires write permissions."
1109 But since the files are already owned by the web server user, and already can't be written by anyone else, it seems a bit of a pointless half measure since the owner can override any permissions. Enabling the .htaccess files seems more useful, which it doesn't even mention.
1111 We could also set up a backup for piwik data, but traffic stat data is not that important so I haven't bothered.
1113 == Choosing Extensions ==
1115 Mediawiki.org has pages for ~5200 extensions. Mediawiki maintains ~700 extensions [http://git.wikimedia.org/tree/mediawiki%2Fextensions.git in it's git repo]. Wikipedia uses [https://en.wikipedia.org/wiki/Special:Version over 100 extensions]. Major distributors package [[mediawikiwiki:Comparison_of_extensions_in_distributions| ~36 extensions]]. We looked closely at the distributor's and briefly at the Mediawiki repo extensions. We haven't found any other useful list or recommendations.
1117 Here are brief descriptions of extensions that are part of distributions and why they were rejected for this wiki.
1119 {| class="wikitable"
1121 | '''InputBox''' || Add html forms to pages. Can't imagine using it. Would install if I did.
1123 | '''Pdfhandler''' || Gallery of pages from a pdf file. Can't imagine using it. Would install if I did.
1125 | '''Footnote''' || deprecated in newer versions
1127 | '''NewUserNotif''' || Send me a notification when a user registers. Seems like an excessive notification.
1129 | '''NewestPages''' || A page creation history that doesn't expire like recent-changes. Meh
1131 | '''RSSReader''' || Embed an rss feed. Can't imagine using it. Would install if I did.
1133 | '''Openid''' || Poor UI. 2 pages & 2 links <login> <login with openid> which is confusing & ugly.
1135 | '''Validator''' || dependency of of semantic
1137 | '''Semantic''' || Seems like a lot of trouble around analyzing kinds of data which my wiki will not have.
1139 | '''wikicalendar''' || Make a calendar of events etc. Can't imagine using it. Would install if I did.
1144 ''' Mediawiki Documentation Quality '''
1146 Overall the documentation is good, but like wikipedia, it depends.
1148 The closer a topic is to core functionality and commonly used features, the better the documentation is likely to be. My guess is that Wikimedia Foundation (WMF) has a competing priority of being a good upstream to mediawiki users and being good for their own sites. That, plus the multitude of unconnected extension developers, and official documentation is sometimes neglected in favor of bug reports, readme files, comments, code, and unpublished knowledge. User's edits vary in quality, and often aren't reviewed by anyone. If you run into an issue, try viewing/diffing the most recent version of a page by the last few editors.
1150 One issue is that mediawiki.org needs a lot of organizing, deleting, and verifying of material, and that is relatively unpopular, tedious, and sometimes difficult work. The discussion pages of mediawiki.org are a wasteland of unanswered questions and outdated conversations, which is [https://www.mediawiki.org/wiki/Help:Talk_pages poor form] for a wiki. However, if you communicate well, you can get great help from their [https://www.mediawiki.org/wiki/Communication support forum, irc, and mailing list].
1153 '''Bash here documents, EOF vs 'EOF' '''
1155 Here documents are used throughout this page, some people may not be aware of a small but important syntax. When the delimiter is quoted, as in <<'EOF', then the contents of the here document are exactly verbatim. Otherwise $ and ` are expanded as in bash, and must be quoted by \, which itself must then also be quoted to be used literally.
1158 ''' Mediawiki automation tools survey 7/2014 '''
1161 * https://github.com/ianweller/mw
1162 * http://search.cpan.org/~markj/WWW-Mediawiki-Client/bin/mvs
1163 * https://github.com/alexz-enwp/wikitools 3000 lines of code, no response to a bug reports in 2/2014
1165 Getting basic maintenance
1166 * https://github.com/mwclient/mwclient 2000 lines of code
1168 Actively developed, used by wikimedia foundation a lot.
1169 * [[mediawikiwiki:Manual:Pywikibot]]
1172 ''' Troubleshooting Errors '''
1174 If mediawiki fails to load, or shows an error in the browser, enable some settings and it will print much more useful information. [[mediawikiwiki:Manual:How to debug]]
1178 This page and this wiki is licensed under cc-by-sa 4.0.
1179 This means the code is compatible with gplv3.
1181 == todo list for this page ==
1183 * Evaluate any extensions bundled in 27 which I haven't checked out before.
1185 * Don't require registration for edits