# Sync large quantities of files or directories between local machine and server with rsync

A guide to syncing files between local and remote environment with rsync.

`rsync` is an awesome tool, it’s designed to effortlessly move large quantities
of files, locally or over the network.

If your `rsync` is out of date, I have a
[little post](/update-rsync-on-osx-with-homebrew) on how to upgrade it.

## rsync syntax

```
rsync options source destination
```

Commonly used options:

`-r`

recursive, for moving directories.

`-z`

compress file data.

`-a`

archive mode, combo of `-rlptgoD`, meaning: preserves symbolic
    links, special and device files, modification times, group, owner, and
    permissions.

`-P`

Combo of `--progress` and `--partial`, meaning it
    shows a progress bar and it’s possible to resume interrupted transfers.

`--delete`

Delete files that don't exists in source system, good when syncing.

The usual set of option is `-zaP`, which is easy to remember.

Basic example:

```
$ rsync -zaP --delete ./uploads -p 5555 bob@yourserver.com:/var/www/example.com/public_html
```

That’ll sync a local directory `uploads/` to remote
`/var/www/example.com/public_html`. Local path can be relative.

More examples below.

> **Noteworthy thing**: if you do commands like that a lot, you could put your
> ssh credentials into the `~/.ssh/config` file, then the host can be referred
> with a short name,
> [here's an article on how to do that](/how-to-setup-and-manage-multiple-ssh-keys).

## Moving files around locally

Make some test files:

```shell
$ mkdir rsync-test-{1..2}
$ touch rsync-test-1/file{1..50}
```

Syncing files from `rsynctest1/` to `rsynctest2/` is as follows:

```shell
$ rsync -r rsynctest1/ rsynctest2
```

Notice the forward slash `/` after the source directory. This means that grab
the contents of the directory, not the directory itself. Test the difference if
you like:

```shell
$ rsync -r rsynctest1 rsynctest2
```

But really, the `-a` option would fit here like a fist in the eye:

```shell
$ rsync -a rsynctest1/ rsynctest2
```

It preserves all the setting the directory has, symlinks, permissions etc.

## Sync directory from local to remote

Example syntax for moving a file to a server (assumes you set server
[shortname in ssh config](/how-to-setup-and-manage-multiple-ssh-keys)):

```shell
$ rsync -zaP --delete uploads/ server_shortname:/path/to/uploads
```

**Note:** the `--delete` option removes the files from the remote that are not
present locally, it's truly synchronizing the directories.

**Btw:** tab completion in the remote server is possible,
[see this post for more info](/copy-files-from-local-to-remote-and-tab-complete-in-the-remote-server).

## Sync directory from remote to local

Just "flip it and reverse it":

```shell
$ rsync -zaP --delete server_shortname:/path/to/uploads uploads/
```

## Sync both ways

Run it twice using the `-u` flag.

`-u`, `--update`

Skip files that are newer on the receiver

```shell
$ rsync -ur uploads/ server_shortname:/path/to/uploads
$ rsync -ur server_shortname:/path/to/uploads uploads/
```

With more options:

```shell
$ rsync -zuaP uploads/ server_shortname:/path/to/uploads
$ rsync -zuaP server_shortname:/path/to/uploads uploads/
```

[Or use Unison](http://www.cis.upenn.edu/~bcpierce/unison/download/releases/stable/unison-manual.html),
it’s a tool made just for that, but I’ve never used it myself.

## Moving a shit ton of files

The `-z` option enables compression, **but only for individual files**. It won't
compress the whole directory, which would be ideal if moving thousands of files
for the first time.

To really bulldoze files over to a server, you could go like:

```shell
$ tar zcvf - ./source | ssh bob@example.com "cd /target/dir; tar xvzf -"
```

Here’s what that does:

1. Compresses the source directory
2. Uploads it to the target path
3. Decompresses the tarball in the remote server

If you want it shorter and more usable, wrap it to a function and put it to your
`.bashrc` or `.bash_profile`, then you got it at your disposal anywhere:

```bash
function zipSync() {
  local source=$1
  local target=$2

  # Get the host and path from the $target string
  IFS=':' read -a array <<< "$target"
  local host=${array[0]}
  local destination=${array[1]}

  tar zcvf - $source | ssh $host "cd $destination; tar xvzf -"
}
```

Example:

```shell
$ zipsync uploads my_server:/var/www/example.com/public_html
```

Problems with that function:

- Can’t tab complete.
- Can’t specify port in the target path, this won’t work:
  `zipsync uploads -p 5555 bob@xmpl.com:/path/` cause the `-p` is read as the
  second parameter.

But:

![A stick figure man doing a little jump with hands above their head](/images/its-something.jpg)