How to Use Rsync for Backup and Synchronization

Edited 4 weeks ago by ExtremeHow Editorial Team

BackupCommand LineFilesSynchronizationData ManagementUtilitiesScriptingSecurityStorageTransfers

Rsync is a powerful tool that can be used to create backups and synchronize files across different systems. It is a command-line utility that enables fast and efficient file transfers. By using delta encoding, Rsync only transfers the differences between the source and destination directories. This makes it an ideal tool for backup and synchronization, especially over network connections, due to its low bandwidth consumption.

Understanding the basics of rsync

Rsync can be used to transfer files locally as well as on a remote host. The basic syntax of the rsync command is:

rsync [options] source destination

Here, source is the location of the files you want to transfer, and destination is the location where you want to move them.

Installation

Rsync usually comes pre-installed on most Linux distributions. To check if it is installed, you can type:

rsync --version

If Rsync is not installed, you can add it using the package manager associated with your operating system. For example, on a Debian-based system, you would use:

sudo apt-get install rsync

On Red Hat-based systems, you can install it using:

sudo yum install rsync

Basic use of rsync

The primary function of rsync is the synchronization of files and directories. For example, to copy all files from source_directory to destination_directory, you can use the following command:

rsync -avzh /path/to/source_directory/ /path/to/destination_directory/

Let us analyze the options used in this command:

• -a: Archive mode, which preserves permissions, timestamps, symbolic links, and other special attributes of files.
• -v: Detailed mode, which displays progress during the transfer.
• -z: Compress files during transfer, which can reduce bandwidth usage.
• -h: Human-readable, which makes the output more readable by showing the file size in an easy to understand format.

Note that the final slash in source_directory/ ensures that only the contents of the directory are copied, not the directory itself.

Rsync over SSH

One of the most powerful features of rsync is its ability to sync files over the network using SSH. To ensure secure data transfer, SSH encrypts the data so that no eavesdroppers can read it. To synchronize directories between local and remote systems, you can use:

rsync -avz -e "ssh" user@remote_host:/path/to/remote_directory/ /path/to/local_directory/

This command uses -e option to specify the remote shell program (SSH in this case).

Similarly, to copy files from the local system to the remote server, change source and destination arguments:

rsync -avz /path/to/local_directory/ user@remote_host:/path/to/remote_directory/

If your SSH uses a different port (other than the default 22), you can specify it using:

rsync -avz -e "ssh -p 2222" /path/to/local_directory/ user@remote_host:/path/to/remote_directory/

Handling deletions

When synchronizing directories, you want the destination directory to exactly mirror the source directory, including deletions. The --delete option can be used for this purpose:

rsync -avz --delete /path/to/source_directory/ /path/to/destination_directory/

With --delete, Rsync will delete any files on the destination that do not exist on the source.

Handling large file sets

For large files or a large number of files, Rsync can take a long time to run. You can speed up this process by running Rsync as a background process using & symbol. This allows you to use your command line for other tasks:

rsync -avz /path/to/source/ /path/to/destination/ &

Checksums save time

By default, Rsync decides whether a file should be transferred based on checks such as size and modification time. To enforce a transfer based on content differences, you can use:

rsync -avzc /path/to/source/ /path/to/destination/

The -c option will let Rsync perform a checksum comparison, which may take longer, but ensures that each file transmitted is identical.

Specific file types

You can tell Rsync to include or exclude specific file types using the include and except options. For example, to sync only text files, you could type:

rsync -avz --include '*.txt' --exclude '*' /path/to/source/ /path/to/destination/

Here, --include tells Rsync to only include .txt files, and --exclude tells Rsync to omit everything else.

Backup files progress logging

During a backup operation, it may be important to know what was transferred and when. Rsync has an option for this:

rsync -avz --log-file=rsync.log /path/to/source/ /path/to/destination/

This command creates or appends logs to the rsync.log file.

Advanced uses

For users who need more than the basic features, Rsync provides advanced options. For example, you can set bandwidth limits using the --bwlimit option, which allows you to set the maximum speed of your transfers:

rsync -avz --bwlimit=1000 /path/to/source/ /path/to/destination/

This will limit the bandwidth to 1000 KB/s.

Running like a daemon

Rsync can operate as a daemon, running as a background service to which clients can connect for large setups or script-driven tasks. To enable this, some configuration is required in the rsyncd.conf file. For example:

[backup] path = /path/to/share comment = Backup Directory read only = no list = yes uid = root gid = root auth users = username secrets file = /path/to/rsyncd.secrets

In daemon mode, Rsync listens for requests without the need for SSH. However, be sure to secure this setup with a firewall and authentication methods.

Automation

For regular backups, Rsync can be automated using cron jobs in Linux. This ensures that backups happen at the specified time without any manual input. For example, to set up a daily backup at 2 AM, edit the crontab file using the following:

crontab -e

Add the following line:

0 2 * * * rsync -avz /path/to/source/ /path/to/destination/

This cron job will execute the Rsync command every day at 2 AM.

Conclusion

Rsync is a versatile tool that is invaluable for systematic backup and synchronization tasks. Using the proper options and configuration, you can customize Rsync to meet almost any data backup need. Its ability to perform incremental backups by syncing only changes makes it efficient in terms of both time and network resource usage. The open-source nature and cross-platform support makes Rsync an ideal choice for automated backup solutions.

  Share This Article

If you find anything wrong with the article content, you can request an update