Backing up files with Ruby and Amazon's S3 Service

Posted by Daniel on 04/02/2009

Always have a backup plan, and keep it cheap

I've been putting this off for a while, but I finally took the time to write a backup script for my Ubuntu webserver. Client priorities come first. There are basically three types of files that I need to backup.

1. SQL data
2. Dynamically added assets
3. Git repositories

Since this isn't a lot of data, and everything is stored in flat files (I use SQLite, as the data sets will stay super small), and I set up a directory structure to where all of the above files are under the /var/www directory, everything is super easy. But, where to backup to? Amazon S3 to the rescue.

Amazon S3 to the rescue!

Amazon's S3 service is super cheap, and really easy to use. It's target customer base is developers, not desktop users, so using a CLI-esk method of uploading/downloading files works well. You can't beat $0.15 per GB per month for storage. This means you don't have to worry about availability or any of that horribleness.

Ruby has a sweet library for accessing S3 Buckets, so I'm using that. Find info here: AWS::S3. Install it with:

jimbob@jimbob.com:~$ sudo gem install aws-s3

I've created an executible ruby script:

#!/usr/local/bin/ruby
require 'rubygems'
require 'aws/s3'

path = '/var/www/'

zipped = '/tmp/backup' + Time.now.to_i.to_s + ".tar"

#zip the dirs

system("tar -cvf #{zipped} #{path}")


#s3 Connection
AWS::S3::Base.establish_connection!(
  :access_key_id  => '123',
  :secret_access_key => '1234'
)

bucketName = "yourUniqueBucketName"
bucket = AWS::S3::Bucket.find bucketName

#delete everything in the bucket, unless you want to keep more than one copy
bucket.object_cache.each do |o|
  o.delete
end

#upload the tar
AWS::S3::S3Object.store(zipped, open(zipped), bucketName)


#remove the tmp zipped file
system("rm -rf #{zipped}")

You can now do with it as you please. I've made the script executable, placed it in my system path, and added a cron entry that looks like this:

01 01 * * 0 root backup >> /where/you/want/it/backup.log

Backup made easy!