How to Exclude Files and Directories with Rsync

Introduction

Before we begin talking about how to exclude files and directories with Rsync, let's briefly understand – What is rsync?

rsync is a powerful command-line tool for synchronizing files and directories between two sites using a remote shell. You may replicate data, make incremental backups, and copy files between systems with rsync.

It is known for its efficient use of network resources and ability to preserve file permissions and metadata during synchronization. You may want to exclude one or more files or folders from a copy of data, depending on their name or location.

In this tutorial, you will exclude files and directories with Rsync. We will also address a few FAQs on how to exclude files and directories with Rsync.

Advantages of rsync

1. Efficient data transfer: Rsync uses delta transfer algorithm to transfer only the differences between source and destination, minimizing bandwidth usage.
2. Incremental backups: It supports incremental backup functionality, enabling efficient syncing of large and frequently changing data sets.
3. Network-friendly: Rsync works well over networks, optimizing data transfer and handling interrupted connections.
4. Versatile: It can synchronize files locally or remotely, and supports copying, mirroring, and advanced filtering options.
5. Preservation of permissions and metadata: Rsync can preserve file permissions, timestamps, and other metadata during synchronization, ensuring data integrity.

Before You Begin

You should have a basic understanding of how rsync functions.

We'll use rsync with the -a option in the following examples. This instructs rsync to recursively sync directories, transfer special and block devices, and maintain symbolic links, modification times, group, ownership, and rights.

You must use relative paths to the source directory when excluding files or folders.

To indicate the files and folders you want to exclude, you have two options:

• Using the --exclude option from the command line.
• Using the --exclude-from option from a file.

Exclude a Specific File

To exclude a specific file, use the --exclude option with the relative path to the file.

The file src_directory/file.txt will not be transferred in the following example:

rsync -a --exclude 'file.txt' src_directory/ dst_directory/

Exclude a Specific Directory

Excluding a specific directory is the same as excluding a file; simply use the --exclude option to pass the relative path to the directory, as demonstrated below:

rsync -a --exclude 'dir1' src_directory/ dst_directory/

Use dir1/* instead of dir1: to exclude directory content but not the directory itself.

rsync -a --exclude 'dir1/*' src_directory/ dst_directory/

Exclude Multiple Files or Directories

To exclude numerous files or folders, Simply use the --exclude options to specify various alternatives:

rsync -a --exclude 'file1.txt' --exclude 'dir1/*' --exclude 'dir2' src_directory/ dst_directory/

If you'd rather use a single --exclude option, put the files and directories you want to exclude in curly braces {} and separate them with a comma, as seen below:

rsync -a --exclude={'file1.txt','dir1/*','dir2'} src_directory/ dst_directory/

Instead of using several --exclude options, you can specify the files and directories you wish to exclude in a file and send the file to the --exclude-from option if the number of files and/or directories you want to exclude is enormous.

The command below performs the same function as the one above:

rsync -a --exclude-from='exclude-file.txt' src_directory/ dst_directory/

file1.txt
dir1/*
dir2

Exclude Multiple Files or Directories Based on a Pattern

You may also use rsync to exclude files and directories based on a pattern that matches the name of the file or directory.

To exclude all .jpg files, for example, you would type:

rsync -a --exclude '*.jpg*' src_directory/ dst_directory/

Excluding all other files and directories except those that meet a specific pattern is a little difficult. Assume you wish to exclude all files and directories other than those ending in .jpg.

Use the following command as an example:

rsync -a -m --include='*.jpg' --include='*/' --exclude='*' src_directory/ dst_directory/

When several include/exclude options are used, the first matching rule takes precedence.

• --include='.jpg' - First, all.jpg files are included.
• --include='*/' - Then all directories in the src directory are included. rsync will only copy *.jpg files in the top-level directory if you don't provide this.
• -m - Removes any directories that are empty.

Another way is to use rsync to sync the output of the search command:

find src_directory/ -name "*.jpg" -printf %P\\0\\n | rsync -a --files-from=- src_directory/ dst_directory/

• -printf %P\\0\\n - removes the src_directory/ directory from the file path.
• --files-from= - specifies that only files from the standard input should be used (files passed from the find command)

FAQs to Exclude Files and Directories with Rsync

Conclusion

You learned how to use rsync to exclude files and directories when transferring data in this tutorial. The rsync User's Manual page has a lot more information about Rsync.

If you have any queries, please leave a comment below and we’ll be happy to respond to them.