Using OpenStack Swift as a backup server for your personal computer
For some of the newer devices equipped with less storage, at least compared to the immense storage requirements nowadays, finding the storage space for your rather sizable backup becomes harder and harder. Some of you might have noticed that our Fuga Cloud platform comes with a rather copious amount of storage and have wondered “Why can’t I use that storage for my rather sizable back-up?”.
If this was the case, then wonder no longer for this handy tutorial will describe the process* in which we will leverage the storage potential of Fuga Cloud to ensure your data is safely stored on your Volume.
[*] Provided that you are working on a Unix machine (like Linux or MacOS), sorry, no Windows. Also, if you are allergic to the command line, this tutorial might be a little bit hard to digest.
Step 1, getting your Fuga Cloud credentials
First head over to horizon and follow the handy pictures below:
For OpenStack Ocata, go to the API Access panel, which is underneath the Overview group.
For OpenStack Mitaka, go to the Access and Security panel underneath the Overview group and find the API Access tab.
Then Click the “Download OpenStack RC File v2.0”, which is on the top-right of the page, this will download a file which includes all the information needed for accessing the Swift storage.
# Note! Whenever the RC file is loaded it will ask for your FUGA password. # This might become troublesome if you load it often or if you want to use an automated backup scheme. # To change this alter the following in the file after you have opened it using your favorite editor (it's your choice, I won't judge... **cough** Vim **cough**): # 1: Remove line 32: echo "Please enter your OpenStack Password for project $OS_TENANT_NAME as user $OS_USERNAME: " # 2: Remove line 33: read -sr OS_PASSWORD_INPUT # 3: Change line 34 from: export OS_PASSWORD=$OS_PASSWORD_INPUT to export OS_PASSWORD= Your FUGA Password
With the RC good to go, we now need the application, in this case, we use Restic (restic.readthedocs.io).
Step 2: Installing Restic
How you get Restic on your system differs per OS, and if yours is among the following you are in luck as this is a whole lot easier
Step 2.1: Using a binary
# Apple MacOS $ brew install restic
# Arch Linux $ pacaur -S restic-git
# Nix & XixOS $ nix-env --install restic
# Debian $ apt-get install restic
Step 2.2: Installing manually
If yours wasn’t there then we’ll compile it ourselves.
Step 2.2.1: Install Go
As Restic is written in Go, we’ll need the Go language, paste the following into your console and press enter to execute the command:
$ wget https://storage.googleapis.com/golang/go1.9.2.linux-amd64.tar.gz $ sudo tar -xvf go1.9.2.linux-amd64.tar.gz $ sudo mv go /usr/local $ export GOROOT=/usr/local/go $ export GOPATH=$HOME/go $ export PATH=$PATH:$GOROOT/bin $ go version
If all went well you should now have some text on your console informing you that Go version 1.9.2 has successfully been installed.
Step 2.2.2: Compiling Restic
Next, clone the git repository of Restic and compile it.
$ git clone https://github.com/restic/restic $ cd restic $ go run build.go
Step 3: Moving restic to /bin
If it successfully compiled you should have a file named Restic in your current directory, move this to the folder which holds the system binaries so it can be easily executed from anywhere.
$ sudo mv restic /bin
If this completes, typing “restic” into the console and pressing enter should return a large amount of text telling you how to use the restic command. If it doesn’t, try closing and reopening the console.
Step 3: Setting up Restic
The first step is sourcing the RC file we’ve got in step one.
$ source [Name of your RC file].sh
Following this you should be able to run:
restic -r swift:[Name of Container]:/path(path is optional) init
This creates a container in Swift with the name you set in [Name of Container] with a subdirectory named path.
$ restic -r swift:Back_up:/work init
The above code creates a Container named “Back_up” with a subfolder named “work” which is usable as a backup location.
# Note! Running any sort of Restic command which connects to a backup location prompts you for a password, # this can be automated by setting an environment variable RESTIC_PASSWORD, using: $ export RESTIC_PASSWORD="ThisPassword123"
If you add this lines to .bashrc, like above, you don’t have to insert these commands every time you want to backup.
Step 4: Backing up
With the repository set up, it is now only a matter of backing up.
$ restic -r swift:[Name of Container]:/path(path is optional) backup [Folder or file]
By running this command you restore the backup file or folder (you’ve done this in step 4) to the location indicated by –target. Restore requires either the snapshot ID (returned from when you updated the data) or the word “latest” to retrieve the most recent version. Execute the command and sit back while the data is retrieved, mind that unlike the backup Restic doesn’t provide a progress bar but something similar can be achieved by using the following:
Step 5: Restoring
With the data now safely stored in the Swift object store, it is now only a matter of retrieving the data when you need it, using this command:
$ restic -r swift:[Name of Container]:/path(path is optional) restore [id or latest] --target [Folder or file]
By using this command you restore the backup file or folder(you’ve done this in step 4) to the location indicated by –target. Restore requires either the snapshot ID (returned from when you updated the data) or the word “latest” to retrieve the most recent version. Execute the command and sit back while the data is retrieved, mind that unlike the backup Restic doesn’t provide a progress bar but something similar can be achieved by using the following:
$ watch -1n du -s [Folder which is being restored]
This will run in your current console, so I would recommend that you open a second console, to exit the command simply press
CTRL+C to close watch.
Step 6: Rince and repeat
With the knowledge to create repositories, backup and restore your data using Swift, I would suggest that you take a look at the Restic documentation and see what else it can do.
I would suggest looking into Cron a time-based Unix utility to run backups automatically.
I want to know more:
Creating a LAMP instance Launching an instance (VM) on Fuga Cloud is very fast thanks to our SSD-drives. But without the software we need, the instance is nearly useless. Of course we can always install our software manually, but that’s cumbersome and error prone. Thankfully the Openstack platform and Ubuntu support cloud-init which enables us to attach an install-script to our instance. So when we launch our instance in Horizon, our software is automagically installed with it.