I’d like to blog more about those powerful commands that we carefully craft in our day-to-day.
For a file transfer I just ran
$ rsync -a --info=progress2 source dest 17,310,485,259 98% 49.04MB/s 0:05:36 (xfr#87446, to-chk=0/99316)
As you can see in the output, it transferred 99316 files (~17 GB of data) within ~5 minutes.
Re-running the same command immediately again compared the same 99316 files via metadata in a matter of ~2 seconds:
$ time rsync -a --info=progress2 source dest 0 0% 0.00kB/s 0:00:00 (xfr#0, to-chk=0/99316) 0.776u 2.353s 0:02.49 125.3% 422+224k 5+0io 3pf+0w
This was with
rsync version 3.2.5 protocol version 31.
Another example with
--stats to get some aggregate numbers upon command completion:
$ rsync -a --info=progress2 --stats . /mnt/pool2022/jpghome/backup/from-nas-old-pool/audio/last-state 48,401,206,545 76% 103.85MB/s 0:07:24 (xfr#4887, to-chk=0/7692) Number of files: 7,692 (reg: 7,101, dir: 591) Number of created files: 5,202 (reg: 4,887, dir: 315) Number of deleted files: 0 Number of regular files transferred: 4,887 Total file size: 63,275,198,202 bytes Total transferred file size: 48,401,206,545 bytes Literal data: 48,401,206,545 bytes Matched data: 0 bytes File list size: 218,538 File list generation time: 0.019 seconds File list transfer time: 0.000 seconds Total bytes sent: 48,413,582,133 Total bytes received: 97,026 sent 48,413,582,133 bytes received 97,026 bytes 108,429,292.63 bytes/sec total size is 63,275,198,202 speedup is 1.31
There’s quite a bit of specific functionality defined by
-a is the archive mode which really is representing a specific permutation of options:
archive mode is -rlptgoD (no -A,-X,-U,-N,-H)
The full specification for
--info isn’t even in the man page, it’s provided by running
± rsync --info=help Use OPT or OPT1 for level 1 output, OPT2 for level 2, etc.; OPT0 silences. BACKUP Mention files backed up COPY Mention files copied locally on the receiving side DEL Mention deletions on the receiving side FLIST Mention file-list receiving/sending (levels 1-2) MISC Mention miscellaneous information (levels 1-2) MOUNT Mention mounts that were found or skipped NAME Mention 1) updated file/dir names, 2) unchanged names NONREG Mention skipped non-regular files (default 1, 0 disables) PROGRESS Mention 1) per-file progress or 2) total transfer progress REMOVE Mention files removed on the sending side SKIP Mention files skipped due to transfer overrides (levels 1-2) STATS Mention statistics at end of run (levels 1-3) SYMSAFE Mention symlinks that are unsafe ALL Set all --info options (e.g. all4) NONE Silence all --info options (same as all0) HELP Output this help message Options added at each level of verbosity: 0) NONREG 1) COPY,DEL,FLIST,MISC,NAME,STATS,SYMSAFE 2) BACKUP,MISC2,MOUNT,NAME2,REMOVE,SKIP
rsync flags are valuable read about and to choose carefully. In 2014, I started building a backup solution for myself involving the following powerful flags — they haven’t let me down until today:
rsync_cmd = [ "/usr/local/bin/rsync", "--archive", "--verbose", "--hard-links", "--delete", "--fuzzy", "--stats", self._source_dir, self._target_dir, ]
Note that sometimes you also find
--flags=name0 being recommended. This makes sense combination with
-v (verbose). Then,
--flags=name0 prevents each individual file name from being emitted to the terminal.