How the lite database backup is created

How the lite database backup is created


A "lite" version of the nightly Phoenix database backup is created each night, as the full database backup size has grown to be too big to easily work with (it takes a long time to download and import to developers' machines).

The lite backup file is /home/backup/phoenix_lite.sql.gz and is currently created and stored on the slave database server.

Creation process

The scripts for creating the lite database backup can be found in /root/rackspace-api/ on the slave database sever. At the time of writing, these scripts don't exist in git. The archive "make-db-lite.tar" attached to this page contains a copy of the scripts, as of the upload date for that file.

The process is:

  • The slave database server creates a temporary VM on the Rackspace public cloud, based on a saved VM image.
  • The slave database passes the full database backup to the VM, which imports it into MySQL.
  • A series of SQL commands are run on the VM's copy of the database, to delete much of the old data. This is what makes the database smaller.
  • A mysqldump is performed on the VM and this is gzipped. This is the lite backup file.
  • The lite backup file is copied back to the slave database server.
  • The VM is deleted.

Cron job and log file

The creation process is triggered each night on the root cron on the slave database server and the output is logged to the file /var/log/make-backup-lite. On 19th March 2016, the process takes around 2.5 hours to complete.

blog tag: