rsync – A versatile tool for backup

rsync is a software that help us in synchronising data between two locations. It just transfers the difference and not whole file.

For example asssume that I want to keep my home directory properly backed up say in /remote/backup. Easiest way of doing this is

$ cp -r myhome /remote/backup/myhome

However, with cp every file is copied, may be I can use -u flag so that only newer files are copied. Still whole file is copied. If the files are large then it takes time and unnecessary i/o.

rsync is a handy tool in such situations. For the above task run

$ rsync myhome/ /remote/backup/myhome

Now rsync will compare contents of myhome and /remote/backup/myhome and transfer only the changes.

rsync is capable of using ssh or rsync server to transfer to remote machines.

e.g $ rsync -e ‘ssh -l ram’ myhome/ remotemachine:/backup/myhome

This will transfer data from myhome to remotemachine. If you have passswordless login on the remote system, backing up data to a remote system looks like a local command.

Here is some interesting performance statistics about rsync

Source /var destination: /backup both on same  machine
Size /var : 923M
First time creation (/backup/var doesn’t exist)
$time rsync /var/ /backup/

sent 892100986 bytes received 269506 bytes 4971423.35 bytes/sec
total size is 890990005 speedup is 1.00

real 2m58.480s
user 0m13.933s
sys 0m14.885s

During this the cpu and memory usage
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
9793 root 20 0 21004 2008 488 S 9 0.2 0:02.30 rsync
9791 root 20 0 21116 2420 896 D 9 0.2 0:02.24 rsync

Note how cpu intensive it is – 9%. It goes up if files are large.

Running the same after about 15 minutes – to allow /var to change sufficiently

$time rsync /var/ /backup/
sent 18327952 bytes received 9121 bytes 5239163.71 bytes/sec
total size is 890930136 speedup is 48.59

real 0m2.616s
user 0m0.588s
sys 0m0.836s

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
10213 root 20 0 3728 1128 644 R 5 0.1 0:00.16 rsync
10212 root 20 0 8432 1576 872 D 5 0.2 0:00.14 rsync

Now the changes transferred in 2.6 s! About 18Mb transferred. That is the good thing about rsync.

Update

31 Aug 09

Note: I have updated my post to use /backup instead of /tmp. rsync modifies file permissions from source. Using tmp might land you in trouble. Apology for not taking care before.

2 Responses to rsync – A versatile tool for backup

  1. ambi says:

    Can you do a hot backup of a database with rsync ?

    • ramanchennai says:

      So far I tried rsync with sybase and postgres. I didn’t find any problem. I could run database with copied version. Normally I use this technique for cloning my production system and use it as a test system.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: