Commands

How to use rsync (Remote Sync) in Linux

Rsync (Remote Sync) is a most commonly used command for copying and synchronizing files and directories remotely as well as locally in Linux/Unix systems.  It provides fast incremental file transfer by transferring only the differences between the source and the destination.
Rsync can be used for mirroring data, incremental backups, copying files between systems and as a replacement for scp, sftp, and cp commands for everyday use.
In this article we will discuss 13 useful rsync command examples in Linux, these examples will help specially Linux beginners to manage their sync, mirroring, and backup task more efficiently.

Installing Rsync

The rsync utility is pre-installed on most Linux distributions. You can check if it is installed on your system by typing:

$ rsync --version

Example output:

rsync  version 3.1.2  protocol version 31
Copyright (C) 1996-2015 by Andrew Tridgell, Wayne Davison, and others.
Web site: http://rsync.samba.org/
Capabilities:
    64-bit files, 64-bit inums, 64-bit timestamps, 64-bit long ints,
    socketpairs, hardlinks, symlinks, IPv6, batchfiles, inplace,
    append, ACLs, xattrs, iconv, symtimes, prealloc

rsync comes with ABSOLUTELY NO WARRANTY.  This is free software, and you
are welcome to redistribute it under certain conditions.  See the GNU
General Public Licence for details.

If you don’t have rsync installed on your system, you can easily install it using your distribution’s package manager.

Install Rsync on Ubuntu and Debian

$ sudo apt install rsync -y

Install Rsync on CentOS and Fedora

$ sudo yum install rsync

Rsync Command Syntax

Before going into how to use the rsync command, let’s start by reviewing the basic syntax.

Local Sync: # rsync {options} {Source} {Destination}
Remote Sync pull: # rsync {options}  <User_Name>@<Remote-Host>:<Source-File-Dir>  <Destination>
Remote Sync Push: # rsync  <Options>  <Source-Files-Dir>   <User_Name>@<Remote-Host>:<Destination>

Some of the commonly used options in rsync command are listed below:

  • -v, –verbose                             Verbose output
  • -q, –quiet                                  suppress message output
  • -a, –archive                              archive files and directory while synchronizing ( -a equal to following options -rlptgoD)
  • -r, –recursive                           sync files and directories recursively
  • -b, –backup                              take the backup during synchronization
  • -u, –update                              don’t copy the files from source to destination if destination files are newer
  • -l, –links                                   copy symlinks as symlinks during the sync
  • -n, –dry-run                             perform a trial run without synchronization
  • -e, –rsh=COMMAND            mention the remote shell to use in rsync
  • -z, –compress                          compress file data during the transfer
  • -h, –human-readable            display the output numbers in a human-readable format
  • –progress                                 show the sync progress during transfer

Let’s jump into the useful examples of rsync command.

1. Copy/Sync files and directory locally

Copy/Sync a File on a Local Computer

This following command will sync a single file on a local machine from one location to another location. For example, a file name backup.tar needs to be copied or synced to /tmp/backups/ folder.

$ rsync -zvh backup.tar /tmp/backups/

Example output:

created directory /tmp/backups
backup.tar
sent 14.71M bytes  received 31 bytes  3.27M bytes/sec
total size is 16.18M  speedup is 1.10

In above we have used the options like -z for compression, -v for verbose output and -h for human readable output, -a for archived.
In above example, you can see that if the destination is not already exists rsync will create a directory automatically for destination.

Copy/Sync a Directory on Local Computer (rsync -zavh)

Let’s assume we want to copy or sync rasho user’s home directory to /opt/backup folder, execute the below rsync command with options -zavh:

$ rsync -zavh /home/rasho /opt/backup

Example output:

sending incremental file list
rasho/
rasho/.bash_logout
rasho/.bash_profile
rasho/.bashrc
rasho/OpenStack-Networking.pdf

sent 4.09M bytes  received 96 bytes  8.19M bytes/sec
total size is 6.15M  speedup is 1.50

2. Copy files & directories recursively locally (rsync -zrvh or rsync -zavh)

Let’s assume we have multiple files and directories inside rasho user home directory, use below rsync command to copy files and directories recursively, either use -a or -r option to copy files and directories recursively.

Note : In rsync command -a option is used for archiving during the copy or sync and apart from archiving -a option is also used for followings:

  • recursively copy files and directory
  • copy symlinks as symlinks
  • preserve permissions
  • preserve group
  • preserve modification time
  • preserve ownership
$ rsync -zrvh /home/rasho /opt/backup
or
$ rsync -zavh /home/rasho/test /opt/backup

3. Copy a Directory from Local Server to a Remote Server

Let’s suppose we want to copy the folder “/home/rasho/test” from local machine to remote machine (192.168.1.88) under the /opt

$ rsync -zarvh /home/rasho/test root@192.168.1.88:/opt
root@192.168.1.88's password:
…………………………………………………………
test/OTRS/
test/OTRS/Database-Selection-OTRS-Installation-CentOS7.jpg
test/OTRS/Install-OTRS-Accept-License.png
test/OTRS/Install-OTRS-CentOS7-RHEL7.jpg
test/OTRS/mysql-secure-installation-centos7-part2.jpg

sent 34.85M bytes  received 2.65K bytes  6.34M bytes/sec
total size is 37.77M  speedup is 1.08
……………………………………………………………

4. Copy/Sync a Remote Directory to a Local Machine

This command will help you sync a remote directory to a local directory. Here in this example, a directory /home/remoteuser/rpmpkgs which is on a remote server is being copied in your local computer in /tmp/myrpms.

$ rsync -avzh root@192.168.1.88:/home/remoteuser/rpmpkgs /tmp/myrpms

root@192.168.1.88's password:

receiving incremental file list
created directory /tmp/myrpms
rpmpkgs/
rpmpkgs/httpd-2.2.3-82.el5.centos.i386.rpm
rpmpkgs/mod_ssl-2.2.3-82.el5.centos.i386.rpm
rpmpkgs/nagios-3.5.0.tar.gz
rpmpkgs/nagios-plugins-1.4.16.tar.gz
sent 91 bytes  received 4.99M bytes  322.16K bytes/sec
total size is 4.99M  speedup is 1.00

5. Specify remote shell during synchronization (rsync -e ssh)

In rsync command we can specify the remote shell like ssh during synchronization, let’s assume we want to use secure communication between source to destination machine via rsync then we should use ssh as remote shell, example is shown below:

$ rsync -zarvh -e ssh  root@192.168.1.88:/home/remoteuser/rpmpkgs  /tmp
root@192.168.1.88's password:
receiving incremental file list
………………………………………………………………………………
rpmpkgs/
rpmpkgs/

rpmpkgs/httpd-2.2.3-82.el5.centos.i386.rpm
rpmpkgs/mod_ssl-2.2.3-82.el5.centos.i386.rpm
rpmpkgs/nagios-3.5.0.tar.gz
rpmpkgs/nagios-plugins-1.4.16.tar.gz

sent 91 bytes  received 4.99M bytes  322.16K bytes/sec
total size is 4.99M  speedup is 1.0

6. Show Progress While Transferring Data with rsync

To show the progress while transferring the data from one machine to a different machine, we can use –progress option for it. It displays the files and the time remaining to complete the transfer.

rsync -avh --progress root@192.168.1.88:/opt/rpmpkgs /tmp
root@192.168.1.88's password:
receiving incremental file list
……………………………………………………………………………………………………..
rpmpkgs/
rpmpkgs/httpd-2.4.6-88.el7.centos.x86_64.rpm
          2.84M 100%   35.22MB/s    0:00:00 (xfr#6, to-chk=18/25)
rpmpkgs/httpd-tools-2.4.6-88.el7.centos.x86_64.rpm
         92.50K 100%    1.13MB/s    0:00:00 (xfr#7, to-chk=17/25)
rpmpkgs/postfix-2.10.1-7.el7.x86_64.rpm
          2.56M 100%   14.44MB/s    0:00:00 (xfr#17, to-chk=7/25)
.........................................

sent 484 bytes  received 16.38M bytes  3.64M bytes/sec
total size is 16.37M  speedup is 1.00

7. Use of –include and –exclude Options

There are some situations where we want to copy or sync files & directories of specific type and want to exclude files or directories of specific type. Rsync command supports both include and exclude options.
In the below example want to copy the files of type pdf and rpm and exclude png file types.

 rsync -avz -e ssh --include '*.pdf *.rpm' --exclude '*.png' /home/rasho/test root@192.168.1.88:/opt
root@192.168.1.88's password:
sending incremental file list
test/
test/OpenStack-Networking.pdf
test/httpd-2.4.6-88.el7.centos.x86_64.rpm
test/httpd-tools-2.4.6-88.el7.centos.x86_64.rpm
test/postfix-2.10.1-7.el7.x86_64.rpm

sent 9,469,912 bytes  received 96 bytes  2,705,716.57 bytes/sec
total size is 11,647,907  speedup is 1.23

8. Sync only the directory structure

There are some scenarios where we want to copy the directory structure skiping files only from local machine to remote or vice versa.
Let’s suppose we want to copy the directory structure of “/home/rasho” from local machine to remote machine (192.168.1.88) under /opt folder.

$ rsync -av -f"+ */" -f"- *" /home/rasho root@192.168.1.88:/opt/
root@192.168.1.88's password:
building file list ... done
………………………………………………
rasho/

sent 43 bytes  received 19 bytes  17.71 bytes/sec
total size is 0  speedup is 0.00

9. Use of –delete Option

If you have already synced files from source to destination and from source you have deleted the files then you can force rsync command to delete the files on destination using the –delete option, example is shown below

$ rsync -avz --delete /opt/rpmpkgs root@192.168.1.88:/tmp/rpmpkgs
root@192.168.1.88's password:
sending incremental file list
deleting rpmpkgs/apr-util-1.5.2-6.el7.x86_64.rpm
deleting rpmpkgs/apr-1.4.8-3.el7_4.1.x86_64.rpm
rpmpkgs/

sent 862 bytes  received 105 bytes  276.29 bytes/sec
total size is 15,947,152  speedup is 16,491.37

10. Set the max size of files to be transferred

You can specify the Max file size to be transferred or sync. You can do it with –max-size option.
Here in this example, Max file size is 200k, so this command will transfer only those files which are equal or smaller than 200k.

# rsync -avzhe ssh --max-size='200k' /var/lib/rpm/ root@192.168.1.88:/root/tmprpm

root@192.168.1.88's password:

sending incremental file list
created directory /root/tmprpm./
Conflictname
Group
Installtid
................

sent 189.79K bytes  received 224 bytes  13.10K bytes/sec
total size is 38.08M  speedup is 200.43

11. Automatically delete source files after successful transfer

Now, suppose you have a main web server and a data backup server, you created a daily backup and synced it with your backup server, now you don’t want to keep that local copy of backup in your web server.
So, will you wait for transfer to complete and then delete those local backup file manually? Of Course NO. This automatic deletion can be done using –remove-source-files option.

$rsync --remove-source-files -zvh backup.tar /tmp/backups/backup.tar
sent 14.71M bytes received 31 bytes 4.20M bytes/sec
total size is 16.18M speedup is 1.10

$ ll backup.tar
ls: backup.tar: No such file or directory

12. Set Bandwidth Limit and Transfer File

You can set the bandwidth limit while transferring data from one machine to another machine with the the help of –bwlimit option. This options helps us to limit I/O bandwidth.

# rsync --bwlimit=100 -avzhe ssh  /var/lib/rpm/  root@192.168.1.88:/root/tmprpm/
root@192.168.1.88's password:
sending incremental file list
sent 324 bytes  received 12 bytes  61.09 bytes/sec
total size is 38.08M  speedup is 113347.05

13. View the difference in files & directories between source and destination

Use -i option in rsync command to list the difference in files and directories between source and destination. Example is shown below

$ rsync -avzi /home/pkumar/test root@192.168.1.88:/opt
root@192.168.1.88's password:
sending incremental file list
.d..t...... test/
<f.st...... techi/projects.txt

sent 438 bytes  received 45 bytes  138.00 bytes/sec
total size is 11,648,064  speedup is 24,116.07

As per above command output, there is difference in file called “projects.txt” on destination. Following are the meaning of the keywords in above output,

  • d: indicates change in destination file
  • f: indicates a file
  • t: indicates change in timestamps
  • s: indicates change in size

That’s all with rsync now, you can see man pages for more options. Stay connected with Lintutfor more exciting and interesting tutorials in future. Do leave your comments and suggestions.

Leave a Reply

Your email address will not be published. Required fields are marked *

CAPTCHA


This site uses Akismet to reduce spam. Learn how your comment data is processed.

Back to top button