Difference between revisions of "BlikBack"

Revision as of 12:49, 29 November 2020

<translate> It is important to make regular backups of the data in your wiki. This page provides an overview of the backup process for a typical MediaWiki wiki; you will probably want to devise your own backup scripts or schedule to suit the size of your wiki and your individual needs.

Overview

MediaWiki stores important data in two places:

Database: Pages and their contents, users and their preferences, metadata, search index, etc.
File system: Software configuration files, custom skins, extensions, images (including deleted images), etc.

Consider making the wiki read-only before creating the backup - see <tvar|readOnly>Template:Ll</>. This makes sure all parts of your backup are consistent (some of your installed extensions may write data nonetheless).

File transfer

You will have to choose a method for transferring files from the server where they are:

Non-private data you can simply [<tvar|url>https://github.com/WikiTeam/wikiteam/wiki/Tutorial#Publishing_the_dump</> publish on archive.org] and/or in a dumps/ directory of your webserver.
SCP (or WinSCP), SFTP/FTP or any other transfer protocol you choose.
The hosting company might provide a file manager interface via a web browser; check with your provider.

Database

Most of the critical data in the wiki is stored in the database. When using the default [[<tvar|1>Special:MyLanguage/Manual:MySQL</>|MySQL or MariaDB backend]], the database can be dumped into a script file which can be used later to recreate the database and all the data in it from scratch. </translate>

MySQL

Automysqlbackup

<translate> See the package on Debian: </translate>

Invalid language.

You need to specify a language like this: <source lang="html4strict">...</source>

Supported languages for syntax highlighting:

4cs, 6502acme, 6502kickass, 6502tasm, 68000devpac, abap, actionscript, actionscript3, ada, algol68, apache, applescript, apt_sources, arm, asm, asp, asymptote, autoconf, autohotkey, autoit, avisynth, awk, bascomavr, bash, basic4gl, bf, bibtex, blitzbasic, bnf, boo, c, c_loadrunner, c_mac, caddcl, cadlisp, cfdg, cfm, chaiscript, cil, clojure, cmake, cobol, coffeescript, cpp, cpp-qt, csharp, css, cuesheet, d, dcl, dcpu16, dcs, delphi, diff, div, dos, dot, e, ecmascript, eiffel, email, epc, erlang, euphoria, f1, falcon, fo, fortran, freebasic, freeswitch, fsharp, gambas, gdb, genero, genie, gettext, glsl, gml, gnuplot, go, groovy, gwbasic, haskell, haxe, hicest, hq9plus, html4strict, html5, icon, idl, ini, inno, intercal, io, j, java, java5, javascript, jquery, kixtart, klonec, klonecpp, latex, lb, ldif, lisp, llvm, locobasic, logtalk, lolcode, lotusformulas, lotusscript, lscript, lsl2, lua, m68k, magiksf, make, mapbasic, matlab, mirc, mmix, modula2, modula3, mpasm, mxml, mysql, nagios, netrexx, newlisp, nsis, oberon2, objc, objeck, ocaml, ocaml-brief, octave, oobas, oorexx, oracle11, oracle8, oxygene, oz, parasail, parigp, pascal, pcre, per, perl, perl6, pf, php, php-brief, pic16, pike, pixelbender, pli, plsql, postgresql, povray, powerbuilder, powershell, proftpd, progress, prolog, properties, providex, purebasic, pycon, pys60, python, q, qbasic, rails, rebol, reg, rexx, robots, rpmspec, rsplus, ruby, sas, scala, scheme, scilab, sdlbasic, smalltalk, smarty, spark, sparql, sql, stonescript, systemverilog, tcl, teraterm, text, thinbasic, tsql, typoscript, unicon, upc, urbi, uscript, vala, vb, vbnet, vedit, verilog, vhdl, vim, visualfoxpro, visualprolog, whitespace, whois, winbatch, xbasic, xml, xorg_conf, xpp, yaml, z80, zxbasic

$ apt show automysqlbackup
[...]
Description: automysqlbackup creates backup every day, week and month for all of your MySQL database, to a configured folder. There's nothing to do but to install this package, and you'll rest assured that you have a way to go back in the history of your database.
[...]

<translate> Install the package: </translate>

Invalid language.

You need to specify a language like this: <source lang="html4strict">...</source>

Supported languages for syntax highlighting:

4cs, 6502acme, 6502kickass, 6502tasm, 68000devpac, abap, actionscript, actionscript3, ada, algol68, apache, applescript, apt_sources, arm, asm, asp, asymptote, autoconf, autohotkey, autoit, avisynth, awk, bascomavr, bash, basic4gl, bf, bibtex, blitzbasic, bnf, boo, c, c_loadrunner, c_mac, caddcl, cadlisp, cfdg, cfm, chaiscript, cil, clojure, cmake, cobol, coffeescript, cpp, cpp-qt, csharp, css, cuesheet, d, dcl, dcpu16, dcs, delphi, diff, div, dos, dot, e, ecmascript, eiffel, email, epc, erlang, euphoria, f1, falcon, fo, fortran, freebasic, freeswitch, fsharp, gambas, gdb, genero, genie, gettext, glsl, gml, gnuplot, go, groovy, gwbasic, haskell, haxe, hicest, hq9plus, html4strict, html5, icon, idl, ini, inno, intercal, io, j, java, java5, javascript, jquery, kixtart, klonec, klonecpp, latex, lb, ldif, lisp, llvm, locobasic, logtalk, lolcode, lotusformulas, lotusscript, lscript, lsl2, lua, m68k, magiksf, make, mapbasic, matlab, mirc, mmix, modula2, modula3, mpasm, mxml, mysql, nagios, netrexx, newlisp, nsis, oberon2, objc, objeck, ocaml, ocaml-brief, octave, oobas, oorexx, oracle11, oracle8, oxygene, oz, parasail, parigp, pascal, pcre, per, perl, perl6, pf, php, php-brief, pic16, pike, pixelbender, pli, plsql, postgresql, povray, powerbuilder, powershell, proftpd, progress, prolog, properties, providex, purebasic, pycon, pys60, python, q, qbasic, rails, rebol, reg, rexx, robots, rpmspec, rsplus, ruby, sas, scala, scheme, scilab, sdlbasic, smalltalk, smarty, spark, sparql, sql, stonescript, systemverilog, tcl, teraterm, text, thinbasic, tsql, typoscript, unicon, upc, urbi, uscript, vala, vb, vbnet, vedit, verilog, vhdl, vim, visualfoxpro, visualprolog, whitespace, whois, winbatch, xbasic, xml, xorg_conf, xpp, yaml, z80, zxbasic

# apt install automysqlbackup

<translate> All your databases will be saved in <tvar|dir>/var/lib/automysqlbackup/</>: </translate>

Invalid language.

You need to specify a language like this: <source lang="html4strict">...</source>

Supported languages for syntax highlighting:

4cs, 6502acme, 6502kickass, 6502tasm, 68000devpac, abap, actionscript, actionscript3, ada, algol68, apache, applescript, apt_sources, arm, asm, asp, asymptote, autoconf, autohotkey, autoit, avisynth, awk, bascomavr, bash, basic4gl, bf, bibtex, blitzbasic, bnf, boo, c, c_loadrunner, c_mac, caddcl, cadlisp, cfdg, cfm, chaiscript, cil, clojure, cmake, cobol, coffeescript, cpp, cpp-qt, csharp, css, cuesheet, d, dcl, dcpu16, dcs, delphi, diff, div, dos, dot, e, ecmascript, eiffel, email, epc, erlang, euphoria, f1, falcon, fo, fortran, freebasic, freeswitch, fsharp, gambas, gdb, genero, genie, gettext, glsl, gml, gnuplot, go, groovy, gwbasic, haskell, haxe, hicest, hq9plus, html4strict, html5, icon, idl, ini, inno, intercal, io, j, java, java5, javascript, jquery, kixtart, klonec, klonecpp, latex, lb, ldif, lisp, llvm, locobasic, logtalk, lolcode, lotusformulas, lotusscript, lscript, lsl2, lua, m68k, magiksf, make, mapbasic, matlab, mirc, mmix, modula2, modula3, mpasm, mxml, mysql, nagios, netrexx, newlisp, nsis, oberon2, objc, objeck, ocaml, ocaml-brief, octave, oobas, oorexx, oracle11, oracle8, oxygene, oz, parasail, parigp, pascal, pcre, per, perl, perl6, pf, php, php-brief, pic16, pike, pixelbender, pli, plsql, postgresql, povray, powerbuilder, powershell, proftpd, progress, prolog, properties, providex, purebasic, pycon, pys60, python, q, qbasic, rails, rebol, reg, rexx, robots, rpmspec, rsplus, ruby, sas, scala, scheme, scilab, sdlbasic, smalltalk, smarty, spark, sparql, sql, stonescript, systemverilog, tcl, teraterm, text, thinbasic, tsql, typoscript, unicon, upc, urbi, uscript, vala, vb, vbnet, vedit, verilog, vhdl, vim, visualfoxpro, visualprolog, whitespace, whois, winbatch, xbasic, xml, xorg_conf, xpp, yaml, z80, zxbasic

$ find /var/lib/automysqlbackup/
/var/lib/automysqlbackup/
/var/lib/automysqlbackup/weekly
/var/lib/automysqlbackup/weekly/my_wiki
/var/lib/automysqlbackup/weekly/my_wiki/my_wiki_week.18.2016-05-07_15h32m.sql.gz
/var/lib/automysqlbackup/monthly
/var/lib/automysqlbackup/daily
/var/lib/automysqlbackup/daily/my_wiki

<translate> Manual backup: </translate>

Invalid language.

You need to specify a language like this: <source lang="html4strict">...</source>

Supported languages for syntax highlighting:

4cs, 6502acme, 6502kickass, 6502tasm, 68000devpac, abap, actionscript, actionscript3, ada, algol68, apache, applescript, apt_sources, arm, asm, asp, asymptote, autoconf, autohotkey, autoit, avisynth, awk, bascomavr, bash, basic4gl, bf, bibtex, blitzbasic, bnf, boo, c, c_loadrunner, c_mac, caddcl, cadlisp, cfdg, cfm, chaiscript, cil, clojure, cmake, cobol, coffeescript, cpp, cpp-qt, csharp, css, cuesheet, d, dcl, dcpu16, dcs, delphi, diff, div, dos, dot, e, ecmascript, eiffel, email, epc, erlang, euphoria, f1, falcon, fo, fortran, freebasic, freeswitch, fsharp, gambas, gdb, genero, genie, gettext, glsl, gml, gnuplot, go, groovy, gwbasic, haskell, haxe, hicest, hq9plus, html4strict, html5, icon, idl, ini, inno, intercal, io, j, java, java5, javascript, jquery, kixtart, klonec, klonecpp, latex, lb, ldif, lisp, llvm, locobasic, logtalk, lolcode, lotusformulas, lotusscript, lscript, lsl2, lua, m68k, magiksf, make, mapbasic, matlab, mirc, mmix, modula2, modula3, mpasm, mxml, mysql, nagios, netrexx, newlisp, nsis, oberon2, objc, objeck, ocaml, ocaml-brief, octave, oobas, oorexx, oracle11, oracle8, oxygene, oz, parasail, parigp, pascal, pcre, per, perl, perl6, pf, php, php-brief, pic16, pike, pixelbender, pli, plsql, postgresql, povray, powerbuilder, powershell, proftpd, progress, prolog, properties, providex, purebasic, pycon, pys60, python, q, qbasic, rails, rebol, reg, rexx, robots, rpmspec, rsplus, ruby, sas, scala, scheme, scilab, sdlbasic, smalltalk, smarty, spark, sparql, sql, stonescript, systemverilog, tcl, teraterm, text, thinbasic, tsql, typoscript, unicon, upc, urbi, uscript, vala, vb, vbnet, vedit, verilog, vhdl, vim, visualfoxpro, visualprolog, whitespace, whois, winbatch, xbasic, xml, xorg_conf, xpp, yaml, z80, zxbasic

# automysqlbackup

<translate> Restore a database: </translate>

gunzip < /var/lib/automysqlbackup/weekly/my_wiki/my_wiki_week.18.2016-05-07_15h32m.sql.gz|mysql -uUSER -pPASSWORD my_wiki

<translate> For other distributions, see on [<tvar|url>https://sourceforge.net/projects/automysqlbackup/</> Sourceforge].

Mysqldump from the command line

The most convenient way to create a dump file of the database you want to back up is to use the standard MySQL dump tool <tvar|mysqldump>mysqldump</> from the command line. Be sure to get the parameters right or you may have difficulty restoring the database. Depending on database size, mysqldump could take a considerable amount of time.

First insert the following line into LocalSettings.php </translate>

$wgReadOnly = 'Dumping Database, Access will be restored shortly';

<translate> this can be removed as soon as the dump is completed.

Example of the command to run on the Linux/UNIX shell: </translate>

mysqldump -h hostname -u userid -p --default-character-set=whatever dbname > backup.sql

<translate> Substituting hostname, userid, whatever, and dbname as appropriate.</translate> <translate> All four may be found in your <tvar|1>Template:Ll</> (LSP) file.</translate> <translate> hostname may be found under <tvar|1>Template:Ll</>; by default it is localhost.</translate> <translate> userid may be found under <tvar|1>Template:Ll</>, whatever may be found under <tvar|2>Template:Ll</>, where it is listed after <tvar|3>DEFAULT CHARSET=</>.</translate> <translate> If whatever is not specified mysqldump will likely use the default of utf8, or if using an older version of MySQL, latin1.</translate> <translate> While dbname may be found under <tvar|1>Template:Ll</>.</translate> <translate> After running this line from the command line mysqldump will prompt for the server password (which may be found under <tvar|1>Template:Ll</> in LSP).</translate>

<translate> See mysqldump for a full list of command line parameters.

The output from mysqldump can instead be piped to gzip, for a smaller output file, as follows </translate>

mysqldump -h hostname -u userid -p dbname | gzip > backup.sql.gz

<translate> A similar mysqldump command can be used to produce XML output instead, by including the --xml parameter. </translate>

mysqldump -h hostname -u userid -p --xml dbname > backup.xml

<translate> and to compress the file with a pipe to gzip </translate>

mysqldump -h hostname -u userid -p --xml dbname | gzip > backup.xml.gz

<translate> Remember to also backup the file system components of the wiki that might be required, e.g., images, logo, and extensions.

Running mysqldump with Cron

Cron is the time-based job scheduler in Unix-like computer operating systems. Cron enables users to schedule jobs (commands or shell scripts) to run periodically at certain times or dates.

A sample command that you may run from a crontab may look like this: </translate>

nice -n 19 mysqldump -u $USER --password=$PASSWORD $DATABASE -c | nice -n 19 gzip -9 > ~/backup/wiki-$DATABASE-$(date '+%Y%m%d').sql.gz

<translate> The nice -n 19 lowers the priority of the process.

Use valid values for $USER, $PASSWORD, and $DATABASE. This will write a backup file with the weekday in the filename so you would have a rolling set of backups. If you want to save the files and extensions as well, you might want to use [[<tvar|backup>User:Flominator/Backup MW</>|this one]]. </translate>

Template:Warning

<translate> If you want to add this task in Cron through Cpanel then you must escape the character "%" </translate>

/usr/bin/mysqldump -u $USER --password=$PASSWORD $DATABASE -c | /bin/gzip > ~/backup/wiki-$DATABASE-$(date '+\%Y\%m\%d').sql.gz

<translate> or you will get an error: </translate>

/bin/sh: -c: line 0: unexpected EOF while looking for matching `''
/bin/sh: -c: line 1: syntax error: unexpected end of file

Tables

Some of the [[<tvar|layout>Special:MyLanguage/Manual:Database layout</>|tables]] dumped have different degrees of temporariness. So to save disk space (beyond just gziping), although those tables need to be present in a proper dump, their data does not. However, under certain circumstances the disadvantage of having to rebuild all this data may outweigh saving disk space (for example, on a large wiki where restoration speed is paramount).

See mailing list thread mysql5 binary schema about the topic.

Latin-1 to UTF-8 conversion

See the [[<tvar|upgrading>Special:MyLanguage/Manual:Upgrading#How do I upgrade from a really old version? In one step, or in several steps?</>|relevant section of the upgrading page]] for information about this process. Also see the [[<tvar|talk>Manual talk:Backing up a wiki</>|talk page]] for more information about working with character sets in general. </translate>

PostgreSQL

<translate> You can use the pg_dump tool to back up a MediaWiki PostgreSQL database. For example: </translate>

pg_dump mywiki > mywikidump.sql

<translate> will dump the mywiki database to mywikidump.sql.

To restore the dump: </translate>

psql mywiki -f mywikidump.sql

<translate> You may also want to dump the global information, e.g. the database users: </translate>

pg_dumpall --globals > postgres_globals.sql

SQLite

Template:Manual:SQLite/Back-up

phpMyAdmin

<translate> Turn your wiki to read only by adding $wgReadOnly = 'Site Maintenance'; to LocalSettings.php.

Open the browser to your phpadmin link, login, choose the wiki database. (Check LocalSettings.php if you're not sure). Select Export. Make sure all items under Export are highlighted, and make sure Structure is highlighted (it's important to maintain the table structure). Optionally check Add DROP TABLE to delete existing references when importing. Make sure Data is checked. Select zipped. Then click on GO and save the backup file.^[1]

Remove $wgReadOnly = 'Site Maintenance'; from LocalSettings.php

Remember to also backup the file system components of the wiki that might be required, e.g. images, logo, and extensions.

External links

For a tutorial, see Siteground: [<tvar|url>https://www.siteground.com/kb/how-to-export-mysql-database/</> MySQL Export: How to backup a MySQL database using phpMyAdmin]</translate>

[<tvar|url>http://web.archive.org/web/20090302015625/http://mambo-manual.org/display/conf/Backing+up+the+Database</> Backing up the Database]

</translate>

HeidiSQL

<translate> [<tvar|url>http://www.heidisql.com/</> HeidiSQL] is similar to phpMyAdmin, but without any restrictions of phpMyAdmin's free version.</translate> <translate> HeidiSQL requires a direct database connection, where some hosts may only offer web interfaces (phpMyAdmin) to firewalled databases.</translate>

File system

MediaWiki stores other components of the wiki in the file system where this is more appropriate than insertion into the database, for example, site configuration files (<tvar|1>Template:Ll</>, <tvar|2>Template:Ll</> (finally removed in 1.23)), image files (including deleted images, thumbnails and rendered math and SVG images, if applicable), skin customisations, extension files, etc.

The best method to back these up is to place them into an archive file, such as a .tar file, which can then be compressed if desired. On Windows, applications such as WinZip or 7-zip can be used if preferred.

For Linux variants, assuming the wiki is stored in /srv/www/htdocs/wiki </translate>

  tar zcvhf wikidata.tgz /srv/www/htdocs/wiki

<translate> It should be possible to backup the entire "wiki" folder in "htdocs" if using XAMPP.

Backup the content of the wiki (XML dump)

It is also a good idea to create an XML dump in addition to the database dump. XML dumps contain the content of the wiki (wiki pages with all their revisions), without the site-related data (they do not contain user accounts, image metadata, logs, etc).^[2]

XML dumps are less likely to cause problems with character encoding, as a means of transferring large amounts of content quickly, and can easily be used by third party tools, which makes XML dumps a good fallback should your main database dump become unusable. </translate>

<translate> To create an XML dump, use the command-line tool <tvar|1>Template:Ll</>, located in the <tvar|2>maintenance</> directory of your MediaWiki installation.</translate> <translate> See <tvar|1>Template:Ll</> for more details.</translate>

<translate> You can also create an XML dump for a specific set of pages online, using Special:Export, although attempting to dump large quantities of pages through this interface will usually time out. </translate>

<translate> To import an XML dump into a wiki, use the command-line tool <tvar|1>Template:Ll</>.</translate> <translate> For a small set of pages, you can also use the Special:Import page via your browser (by default, this is restricted to the sysop group).</translate> <translate> As an alternative to <tvar|1>dumpBackup.php</> and <tvar|2>importDump.php</>, you can use MWDumper, which is faster, but requires a Java runtime environment. </translate>

<translate> See [[<tvar|import>Special:MyLanguage/Manual:Importing XML dumps</>|Manual:Importing XML dumps]] for more information.

Without shell access to the server

</translate> <translate> If you have no shell access, then use the <tvar|WikiTeam>WikiTeam</> Python script <tvar|dumpgenerator>dumpgenerator.py</> from a DOS, Unix or Linux command-line.</translate> <translate> Requires Python v2 (v3 doesn't yet work).</translate>

<translate> To get an XML, with edit histories, dump and a dump of all images plus their descriptions. Without extensions and LocalSettings.php configs. </translate>

python dumpgenerator.py --api=http://www.sdiy.info/w/api.php --xml --images

<translate> Full instructions are at the WikiTeam [<tvar|url>https://github.com/WikiTeam/wikiteam/wiki/Tutorial#I_have_no_shell_access_to_server</> tutorial].

Scripts

</translate> Template:Warning <translate>

[[<tvar|script>Special:MyLanguage/Manual:Backing up a wiki/Duesentrieb's backup script</>|Unofficial backup script]] by User:Duesentrieb.</translate>

Unofficial backup script by Flominator; creates a backup of all files and the database, with optional backup rotation.</translate>

User:Darizotas/MediaWiki Backup Script for Windows - a script for backing up a Windows MediaWiki install. Note: Has no restore feature.</translate>

Unofficial web-based backup script, mw_tools, by Wanglong (allwiki.com); you can use it to back up your database, or use the backup files to recover the database, the operation is very easy.</translate>

[<tvar|wikiteam>https://github.com/WikiTeam/wikiteam</> WikiTeam tools] - if you do not have server access (e.g. your wiki is in a free wikifarm), you can generate an XML dump and an image dump using WikiTeam tools (see [<tvar|backups>https://github.com/WikiTeam/wikiteam/wiki/Available-Backups</> some saved wikis]).

</translate>

Template:Ll

Another [<tvar|url>https://github.com/samwilson/MediaWiki_Backup</> backup script] that: dumps DB, files (just pictures by default, option to include all files in installation), and XML; puts the site into read-only mode; timestamps backups; and reads the charset from LocalSettings. Script does not need to be modified for each site to be backed up. Does not (yet) rotate old backups. Usage: <tvar|code>Template:Nowrap</>. Also provides a script to restore a backup <tvar|code>Template:Nowrap</>.</translate>

Another unofficial {{<tvar|1>ll|Manual:Backing up a wiki/Lanthanis backup CMD</>|MediaWiki backup script for Windows}} by <tvar|2>Lanthanis</> that: exports the pages of specified namespaces as an XML file; dumps specified database tables; and adds further specified folders and files to a ZIP backup file.</translate> <translate> Can be used with Windows task scheduler.</translate>

Script to make periodical backups [<tvar|url>https://github.com/nischayn22/mw_backup</> mw_backup]. This script will make daily, weekly and monthly backups of your database and images directory when run as a daily cron job.

References

</translate>

↑ [[<tvar|talk>Manual_talk:Backing_up_a_wiki#Ubuntu_10.10_-_Step_by_Step_Instructions</>|Manual_talk:Backing_up_a_wiki#Ubuntu_10.10_-_Step_by_Step_Instructions]]
↑ XML dumps are independent of the database structure, and can be imported into future (and even past) versions of MediaWiki.

[[Category:Import/Export{{#translation:}}|BlikBack]] [[Category:MediaWiki for site admins{{#translation:}}]]

[1] [[<tvar|talk>Manual_talk:Backing_up_a_wiki#Ubuntu_10.10_-_Step_by_Step_Instructions</>|Manual_talk:Backing_up_a_wiki#Ubuntu_10.10_-_Step_by_Step_Instructions]]

[2] XML dumps are independent of the database structure, and can be imported into future (and even past) versions of MediaWiki.

[1]

[2]

Difference between revisions of "BlikBack"

Revision as of 12:49, 29 November 2020

Contents

Overview

File transfer

Database

MySQL

Automysqlbackup

Mysqldump from the command line

Running mysqldump with Cron

Tables

Latin-1 to UTF-8 conversion

PostgreSQL

SQLite

phpMyAdmin

External links

HeidiSQL

File system

Backup the content of the wiki (XML dump)

Without shell access to the server

Scripts

See also

References

Navigation menu

Personal tools

Namespaces

Variants

Views

Actions

Search

Navigation

Tools