I’d like to blog more about those powerful commands that we carefully craft in our day-to-day.

For a file transfer I just ran

$ rsync -a --info=progress2 source dest
 17,310,485,259  98%   49.04MB/s    0:05:36 (xfr#87446, to-chk=0/99316)

As you can see in the output, it transferred 99316 files (~17 GB of data) within ~5 minutes.

Re-running the same command immediately again compared the same 99316 files via metadata in a matter of ~2 seconds:

$ time rsync -a --info=progress2 source dest
              0   0%    0.00kB/s    0:00:00 (xfr#0, to-chk=0/99316)   
0.776u 2.353s 0:02.49 125.3%    422+224k 5+0io 3pf+0w

This was with rsync version 3.2.5 protocol version 31.

Another example with --stats to get some aggregate numbers upon command completion:

$ rsync -a --info=progress2 --stats . /mnt/pool2022/jpghome/backup/from-nas-old-pool/audio/last-state
 48,401,206,545  76%  103.85MB/s    0:07:24 (xfr#4887, to-chk=0/7692)   

Number of files: 7,692 (reg: 7,101, dir: 591)
Number of created files: 5,202 (reg: 4,887, dir: 315)
Number of deleted files: 0
Number of regular files transferred: 4,887
Total file size: 63,275,198,202 bytes
Total transferred file size: 48,401,206,545 bytes
Literal data: 48,401,206,545 bytes
Matched data: 0 bytes
File list size: 218,538
File list generation time: 0.019 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 48,413,582,133
Total bytes received: 97,026

sent 48,413,582,133 bytes  received 97,026 bytes  108,429,292.63 bytes/sec
total size is 63,275,198,202  speedup is 1.31

What are -a and --info=progress2 doing?

There’s quite a bit of specific functionality defined by -a and --info=progress2. -a is the archive mode which really is representing a specific permutation of options:

archive mode is -rlptgoD (no -A,-X,-U,-N,-H)

The full specification for --info isn’t even in the man page, it’s provided by running rsync --info=help:

± rsync --info=help
Use OPT or OPT1 for level 1 output, OPT2 for level 2, etc.; OPT0 silences.

BACKUP     Mention files backed up
COPY       Mention files copied locally on the receiving side
DEL        Mention deletions on the receiving side
FLIST      Mention file-list receiving/sending (levels 1-2)
MISC       Mention miscellaneous information (levels 1-2)
MOUNT      Mention mounts that were found or skipped
NAME       Mention 1) updated file/dir names, 2) unchanged names
NONREG     Mention skipped non-regular files (default 1, 0 disables)
PROGRESS   Mention 1) per-file progress or 2) total transfer progress
REMOVE     Mention files removed on the sending side
SKIP       Mention files skipped due to transfer overrides (levels 1-2)
STATS      Mention statistics at end of run (levels 1-3)
SYMSAFE    Mention symlinks that are unsafe

ALL        Set all --info options (e.g. all4)
NONE       Silence all --info options (same as all0)
HELP       Output this help message

Options added at each level of verbosity:
0) NONREG
1) COPY,DEL,FLIST,MISC,NAME,STATS,SYMSAFE
2) BACKUP,MISC2,MOUNT,NAME2,REMOVE,SKIP

rsync flags are valuable read about and to choose carefully. In 2014, I started building a backup solution for myself involving the following powerful flags — they haven’t let me down until today:

        rsync_cmd = [
            "/usr/local/bin/rsync",
            "--archive",
            "--verbose",
            "--hard-links",
            "--delete",
            "--fuzzy",
            "--stats",
            self._source_dir,
            self._target_dir,
            ]

Note that sometimes you also find --flags=name0 being recommended. This makes sense combination with -v (verbose). Then, --flags=name0 prevents each individual file name from being emitted to the terminal.