Rsync is an extremely versatile, powerful, and just downright useful tool to be have. Being able to utilize it properly is extraordinarily useful and will save you hours of work. I have yet to find a good quick-setup guide to get started with basic rsync functionality so here we go.
About Rsync
Rsync is an extremely efficient way to keep data synced between two locations. It works by synchronizing the differences of files instead of copying the whole files over. With rsync one machine acts as the server (which runs the daemon) and the other machines act as clients (they have no configuration associated with them and can sync files just by issuing the rsync command)
Rsync is probably already installed if you use Linux (if not, sudo aptitude install rsync).
Server Config
To start the rsync daemon:
# rsync --daemon
To stop the rsync daemon:
# killall rsync
In order for the server to know what to do, we need a config file located at:
/etc/rsyncd.conf
Here is how mine looks (copy this if you want to get up and running quickly)
motd file = /etc/rsyncd.motd
log file = /var/log/rsyncd.log
pid file = /var/run/rsyncd.pid
lock file = /var/run/rsync.lock
[NAME]
path = /home/<USERNAME>/Desktop/Storage/rsync #this is just an example path
comment = This is a comment
uid = nobody
gid = nogroup #nogroup is for debian, "nobody" will work for most others
read only = no
list = yes
auth users = username #any username you want, just make sure to remember it
hosts allow = 192.168.10.0/24 #this is for allowing access only to your local network
secrets file = /etc/rsyncd.scrt
Now run these commands:
# chmod 755 -R /home/<USERNAME>/Desktop/storage/rsync
# chown -R nobody /home/<USERNAME>/Desktop/storage/rsync
# chmod 600 /etc/rsyncd.scrt
Next we need to add the user we just used for our new setup to /etc/rsyncd.scrt. Edit /etc/rsyncd.scrt so that it looks like this (replace username and password with the actual info)
username:password
Now in order to have rsync start every boot edit /etc/init.d/rsync and change RSYNC_ENABLE=false to true.
Done! The server is now ready to go, just run rsync --daemon to start up the server proccess
Client Command
This is probably the more difficult part of the setup, and that is that you need to know how to construct your rsync command properly. Here are some things to think about:
- Am I going to keep the user permission settings (ie. can only be read/written by root)?
- Am I going to include backup files (.bak)?
- Am I going to include sub folders?
- Am I going to include file links?
--verbose
--progress
--stats
--compress
--rsh=/usr/local/bin/ssh
--recursive
--times
--perms
--links
--delete
--exclude "*bak"
(Notes: delete syncs deletes across syncs ie. I delete a file and sync so it deletes the file on the server. rsh is for encrypting traffic. compress compresses for transit purposes)
Ok, so that list can actually be drastically simplified for most uses (but its good to know what you're doing),
here are the shortcuts:
-a (this combines recursive, symlinks, perms)
-v (verbose)
-h (hardlinks)
-z (compress)
-e ssh (use encryption)
Now take those and put them together based on your needs. Here is how mine looks (syncing pictures on one computer to another):
rsync -avhz --progress --delete /home/user/Pictures/ user@remote.host::NAME/Pictures/
That's it!
DEBUG:
If you get any auth error (or other) here are some things to try-
- make sure you ran chmod 600 and chown nobody and chmod 755 as stated in the server setup instructions
- -e ssh requires a user on the machine with that username (test it by leaving it off to see if it works)
- with: "user@remote.host::NAME/Pictures/" pay attention to NAME which is from the rsyncd.conf in the brackets ("[NAME]") as well as the :: (double colons). try both single ":" and double ":" if you are having trouble.
- Worst case scenario, just use a user that is already on the machine and set the settings to their credentials (not optimal but has a much higher likelihood of working)
