With the advent of the cloud computing, storage has become extremely cheap. We can use this storage to keep backups of our data and keep them safe from any potential data loss disaster. One such extremely cost-effective solution is Amazon S3. Some of the pros of S3 are:

• The storage costs at S3 are very low
• You have virtually unlimited storage capacity
• You only pay for how much you use

In this “how to” we will see how to use Amazon S3 to backup your server data.

Pre-requisites:

• Linux Server with root access (Assumed Debian based)
• An Amazon S3 account i.e. S3 access key and S3 secret key (Sign Up at http://aws.amazon.com/s3/ to get it)

Applications/Softwares Used:

• FUSE Libraries: Filesystem in User Space
• S3FS: Filesystem based on Amazon S3
• Rsync: Remote data synchronization tool

STEPS:

1. Login to your server as root using SSH and install the pre-requisites and dependencies by keying the following command:
   

   Code

   1. apt-get install build-essential libcurl4-openssl-dev libxml2-dev libfuse-dev comerr-dev libfuse2 libidn11-dev libkdb5-4 libkrb5-dev libldap2-dev libselinux1-dev libsepol1-dev pkg-config fuse-utils sshfs rsync mailutils

2. Get a copy of the latest version of S3FS Source, extract it and install it:
   

   Code

   1. wget -c http://s3fs.googlecode.com/files/s3fs-r191-source.tar.gz

   2. tar xvf s3fs-r191-source.tar.gz

   3. cd s3fs

   4. make&&make install

3. To allow non-root users to the fuse drives open /etc/fuse.conf in your favorite editor, find and un-comment user_allow_other
4. Create a bucket for your backups on S3 using the S3 console. In this case we created “server-backup-bucket” as our backup bucket.
5. Create a mount point for our backups and mount our S3 drive:
   

   Code

   1. mkdir -p /mnt/backups/

   2. /usr/bin/s3fs -o allow_other server-backup-bucket -o accessKeyId=XYZ -o secretAccessKey=ABC /mnt/backups/

   
   Now, the backup directory is mounted to /mnt/backups/.
6. Create a shell script inside /opt/backup_scripts/ named mount-s3-backups which uses the above mount method:
   

   Code

   1. #!/bin/bash

   2. IS_MOUNTED=`df | grep -c '/mnt/backups'` # Check if the drive is already mounted

   3. if [ "$IS_MOUNTED" -eq "0" ]; then

   4. /usr/bin/s3fs -o allow_other server-backup-bucketname -o accessKeyId=XYZ -o secretAccessKey=ABC /mnt/backups/

   5. echo "S3 Backup Drive mounted successfully!"

   6. else

   7. echo "S3 Backup Drive already mounted!"

   8. fi

   
   Make this script executable:
   

   Code

   1. chmod +x /opt/backup_scripts/mount-s3-backups

7. Setup a shell script in /opt/backup_scripts/s3-home-backup.sh. In this case I assume we are going to backup the /home/ directory:
   

   Code

   1. #!/bin/bash

   2. modprobe fuse # Loads the Fuse Kernel module

   3. /opt/backup_scripts/mount-s3-backups # Mount S3 Backup directory if not mounted

   4. IS_MOUNTED=`df | grep -c '/mnt/backups'` # Check if the mount was successful

   5. if [ "$IS_MOUNTED" -eq "1" ]; then

   6. TIME_STARTED=`date`

   7. echo "Started Backup..."

   8. echo "Backing up to S3 Drive" > /opt/backup_scripts/tmp.log

   9. echo "START TIME: $TIME_STARTED" >> /opt/backup_scripts/tmp.log

   10. echo "-----------------------------------------" >> /opt/backup_scripts/tmp.log

   11. /usr/bin/rsync -avz --delete /home/ /mnt/backups/home/ >> /opt/backup_scripts/tmp.log # Rsync the /home directory to /mnt/backups/home --delete will ensure deleted file in the filesystem are deleted on the backup.

   12. TIME_ENDED=`date`

   13. echo "-----------------------------------------" >> /opt/backup_scripts/tmp.log

   14. echo "END TIME: $TIME_ENDED" >> /opt/backup_scripts/tmp.log

   15. echo "-----------------------------------------" >> /opt/backup_scripts/tmp.log

   16. mail logs@your-domain.com -s "[S3 Backups] Daily Home Backup Complete" < /opt/backup_scripts/tmp.log # Sends the logs of the backup to your email

   17. umount /mnt/backups # Unmount the Backups Directory

   18. echo "Backup complete!"

   19. else

   20. echo "S3Drive not Mounted" > /opt/backup_scripts/tmp.log

   21. mail logs@your-domain.com -s "[ERROR] S3Drive not Mounted!" < /opt/backup_scripts/tmp.log

   22. fi

   
   We are almost there. Now lets make the /opt/backup_scripts/s3-home-backup.sh executable and run it:
   

   Code

   1. chmod +x /opt/backup_scripts/s3-home-backup.sh

   2. /opt/backup_scripts/s3-home-backup.sh

   
   If everything is OK, the script should run successfully and a mail should be delivered to [email protected] with the logs of the backup.
8. For the last step lets add the script to crontab, open crontab in your favorite editor by running crontab -e and insert the following code:
   

   Code

   1. @daily /opt/backup_scripts/s3-home-backup.sh

   
   The backups interval can be configured by changing the cron interval.

That’s it.. Eight simple steps to securely backup your server data on Amazon S3.

P.S. : Data Backup is a critical operation. Any script that is written must be thoroughly tested before running over the actual data to prevent data loss.