First you’ll need to configure rclone. As the object storage systems
have quite complicated authentication these are kept in a config file
.rclone.conf in your home directory by default. (You can use the
--config option to choose a different config file.)
The easiest way to make the config is to run rclone with the config option:
See the following for detailed instructions for
Rclone syncs a directory tree from one storage system to another.
Its syntax is like this
Syntax: [options] subcommand <parameters> <parameters...>
Source and destination paths are specified by the name you gave the storage system in the config file then the sub path, eg “drive:myfolder” to look at “myfolder” in Google drive.
You can define as many storage paths as you like in the config file.
Copy the source to the destination. Doesn’t transfer unchanged files, testing by size and modification time or MD5SUM. Doesn’t delete files from the destination.
Note that it is always the contents of the directory that is synced, not the directory so when source:path is a directory, it’s the contents of source:path that are copied, not the directory name and contents.
If dest:path doesn’t exist, it is created and the source:path contents go there.
rclone copy source:sourcepath dest:destpath
Let’s say there are two files in sourcepath
This copies them to
If you are familiar with
rsync, rclone always works as if you had
written a trailing / - meaning “copy the contents of this directory”.
This applies to all commands and whether you are talking about the
source or destination.
Sync the source to the destination, changing the destination only. Doesn’t transfer unchanged files, testing by size and modification time or MD5SUM. Destination is updated to match source, including deleting files if necessary.
Important: Since this can cause data loss, test first with the
--dry-run flag to see exactly what would be copied and deleted.
Note that files in the destination won’t be deleted if there were any errors at any point.
It is always the contents of the directory that is synced, not the
directory so when source:path is a directory, it’s the contents of
source:path that are copied, not the directory name and contents. See
extended explanation in the
copy command above if unsure.
If dest:path doesn’t exist, it is created and the source:path contents go there.
Moves the source to the destination.
If there are no filters in use this is equivalent to a copy followed by a purge, but may use server side operations to speed it up if possible.
If filters are in use then it is equivalent to a copy followed by delete, followed by an rmdir (which only removes the directory if empty). The individual file moves will be moved with server side operations if possible.
Important: Since this can cause data loss, test first with the –dry-run flag.
List all the objects in the path with size and path.
List all directories/containers/buckets in the the path.
List all the objects in the the path with modification time, size and path.
Produces an md5sum file for all the objects in the path. This is in the same format as the standard md5sum tool produces.
Produces an sha1sum file for all the objects in the path. This is in the same format as the standard sha1sum tool produces.
Prints the total size of objects in remote:path and the number of objects.
Make the path if it doesn’t already exist
Remove the path. Note that you can’t remove a path with objects in it, use purge for that.
Remove the path and all of its contents. Note that this does not obey
include/exclude filters - everything will be removed. Use
you want to selectively delete files.
Remove the contents of path. Unlike
purge it obeys include/exclude
filters so can be used to selectively delete files.
Eg delete all files bigger than 100MBytes
Check what would be deleted first (use either)
rclone --min-size 100M lsl remote:path rclone --dry-run --min-size 100M delete remote:path
rclone --min-size 100M delete remote:path
That reads “delete everything with a minimum size of 100 MB”, hence delete all files bigger than 100MBytes.
Checks the files in the source and destination match. It compares sizes and MD5SUMs and prints a report of files which don’t match. It doesn’t alter the source or destination.
--size-only may be used to only compare the sizes, not the MD5SUMs.
dedup interactively finds duplicate files and offers to
delete all but one or rename them to be different. Only useful with
Google Drive which can have duplicate file names.
dedupe command will delete all but one of any identical (same
md5sum) files it finds without confirmation. This means that for most
duplicated files the
dedupe command will not be interactive. You
--dry-run to see what would happen without doing anything.
Here is an example run.
Before - with duplicates
$ rclone lsl drive:dupes 6048320 2016-03-05 16:23:16.798000000 one.txt 6048320 2016-03-05 16:23:11.775000000 one.txt 564374 2016-03-05 16:23:06.731000000 one.txt 6048320 2016-03-05 16:18:26.092000000 one.txt 6048320 2016-03-05 16:22:46.185000000 two.txt 1744073 2016-03-05 16:22:38.104000000 two.txt 564374 2016-03-05 16:22:52.118000000 two.txt
$ rclone dedupe drive:dupes 2016/03/05 16:24:37 Google drive root 'dupes': Looking for duplicates using interactive mode. one.txt: Found 4 duplicates - deleting identical copies one.txt: Deleting 2/3 identical duplicates (md5sum "1eedaa9fe86fd4b8632e2ac549403b36") one.txt: 2 duplicates remain 1: 6048320 bytes, 2016-03-05 16:23:16.798000000, md5sum 1eedaa9fe86fd4b8632e2ac549403b36 2: 564374 bytes, 2016-03-05 16:23:06.731000000, md5sum 7594e7dc9fc28f727c42ee3e0749de81 s) Skip and do nothing k) Keep just one (choose which in next step) r) Rename all to be different (by changing file.jpg to file-1.jpg) s/k/r> k Enter the number of the file to keep> 1 one.txt: Deleted 1 extra copies two.txt: Found 3 duplicates - deleting identical copies two.txt: 3 duplicates remain 1: 564374 bytes, 2016-03-05 16:22:52.118000000, md5sum 7594e7dc9fc28f727c42ee3e0749de81 2: 6048320 bytes, 2016-03-05 16:22:46.185000000, md5sum 1eedaa9fe86fd4b8632e2ac549403b36 3: 1744073 bytes, 2016-03-05 16:22:38.104000000, md5sum 851957f7fb6f0bc4ce76be966d336802 s) Skip and do nothing k) Keep just one (choose which in next step) r) Rename all to be different (by changing file.jpg to file-1.jpg) s/k/r> r two-1.txt: renamed from: two.txt two-2.txt: renamed from: two.txt two-3.txt: renamed from: two.txt
The result being
$ rclone lsl drive:dupes 6048320 2016-03-05 16:23:16.798000000 one.txt 564374 2016-03-05 16:22:52.118000000 two-1.txt 6048320 2016-03-05 16:22:46.185000000 two-2.txt 1744073 2016-03-05 16:22:38.104000000 two-3.txt
Dedupe can be run non interactively using the
--dedupe-mode interactive- interactive as above.
--dedupe-mode skip- removes identical files then skips anything left.
--dedupe-mode first- removes identical files then keeps the first one.
--dedupe-mode newest- removes identical files then keeps the newest one.
--dedupe-mode oldest- removes identical files then keeps the oldest one.
--dedupe-mode rename- removes identical files then renames the rest to be different.
For example to rename all the identically named photos in your Google Photos directory, do
rclone dedupe --dedupe-mode rename "drive:Google Photos"
Enter an interactive configuration session.
Prints help on rclone commands and options.
When you are typing commands to your computer you are using something called the command line shell. This interprets various characters in an OS specific way.
Here are some gotchas which may help users unfamiliar with the shell rules
If your names have spaces or shell metacharacters (eg
" etc) then you must quote them. Use single quotes
' by default.
rclone copy 'Important files?' remote:backup
If you want to send a
' you will need to use
rclone copy "O'Reilly Reviews" remote:backup
The rules for quoting metacharacters are complicated and if you want the full details you’ll have to consult the manual page for your shell.
If your names have spaces in you need to put them in
rclone copy "E:\folder name\folder name\folder name" remote:backup
If you are using the root directory on its own then don’t quote it (see #464 for why), eg
rclone copy E:\ remote:backup
Drive, S3, Dropbox, Swift and Google Cloud Storage support server side copy.
This means if you want to copy one folder to another then rclone won’t download all the files and re-upload them; it will instruct the server to copy them in place.
rclone copy s3:oldbucket s3:newbucket
Will copy the contents of
downloading and re-uploading.
Remotes which don’t support server side copy (eg local) will download and re-upload in this case.
Server side copies are used with
copy and will be
identified in the log when using the
Server side copies will only be attempted if the remote names are the same.
This can be used when scripting to make aged backups efficiently, eg
rclone sync remote:current-backup remote:previous-backup rclone sync /path/to/files remote:current-backup
Rclone has a number of options to control its behaviour.
Options which use TIME use the go time parser. A duration string is a possibly signed sequence of decimal numbers, each with optional fraction and a unit suffix, such as “300ms”, “-1.5h” or “2h45m”. Valid time units are “ns”, “us” (or “µs”), “ms”, “s”, “m”, “h”.
Options which use SIZE use kByte by default. However a suffix of
k for kBytes,
M for MBytes and
G for GBytes may be
used. These are the binary units, eg 1, 2**10, 2**20, 2**30
Bandwidth limit in kBytes/s, or use suffix b|k|M|G. The default is
which means to not limit bandwidth.
For example to limit bandwidth usage to 10 MBytes/s use
This only limits the bandwidth of the data transfer, it doesn’t limit the bandwith of the directory listings etc.
The number of checkers to run in parallel. Checkers do the equality checking of files during a sync. For some storage systems (eg s3, swift, dropbox) this can take a significant amount of time so they are run in parallel.
The default is to run 8 checkers in parallel.
Normally rclone will look at modification time and size of files to see if they are equal. If you set this flag then rclone will check the file hash and size to determine if files are equal.
This is useful when the remote doesn’t support setting modified time and a more accurate sync is desired than just checking the file size.
This is very useful when transferring between remotes which store the same hash type on the object, eg Drive and Swift. For details of which remotes support which hash type see the table in the overview section.
rclone --checksum sync s3:/bucket swift:/bucket would run much
quicker than without the
When using this flag, rclone won’t update mtimes of remote files if they are incorrect as it would normally.
Specify the location of the rclone config file. Normally this is in
your home directory as a file called
.rclone.conf. If you run
rclone -h and look at the help for the
--config option you will
see where the default location is for you. Use this flag to override
the config location, eg
rclone --config=".myconfig" .config.
Set the connection timeout. This should be in go time format which
5s for 5 seconds,
10m for 10 minutes, or
The connection timeout is the amount of time rclone will wait for a
connection to go through to a remote object storage system. It is
1m by default.
Mode to run dedupe command in. One of
rename. The default is
interactive. See the dedupe command for more information as to what these options mean.
Do a trial run with no permanent changes. Use this to see what rclone
would do without actually doing it. Useful when setting up the
command which deletes files in the destination.
Using this option will make rclone unconditionally skip all files that exist on the destination, no matter the content of these files.
While this isn’t a generally recommended option, it can be useful in cases where your files change due to encryption. However, it cannot correct partial transfers in case a transfer was interrupted.
Normally rclone will look at modification time and size of files to
see if they are equal. If you set this flag then rclone will check
only the modification time. If
--checksum is set then it only
checks the checksum.
It will also cause rclone to skip verifying the sizes are the same after transfer.
This can be useful for transferring files to and from onedrive which occasionally misreports the size of image files (see #399 for more info).
Using this option will cause rclone to unconditionally upload all files regardless of the state of files on the destination.
Normally rclone would skip any files that have the same
modification time and are the same size (or have the same checksum if
Log all of rclone’s output to FILE. This is not active by default.
This can be useful for tracking down problems with syncs in
combination with the
-v flag. See the Logging section for more
This controls the number of low level retries rclone does.
A low level retry is used to retry a failing operation - typically one
HTTP request. This might be uploading a chunk of a big file for
example. You will see low level retries in the log with the
This shouldn’t need to be changed from the default in normal
operations, however if you get a lot of low level retries you may wish
to reduce the value so rclone moves on to a high level retry (see the
--retries flag) quicker.
Disable low level retries with
This modifies the recursion depth for all the commands except purge.
So if you do
rclone --max-depth 1 ls remote:path you will see only
the files in the top level directory. Using
--max-depth 2 means you
will see all the files in first two directory levels and so on.
For historical reasons the
lsd command defaults to using a
--max-depth of 1 - you can override this with the command line flag.
You can use this command to disable recursion (with
Note that if you use this with
files not recursed through are considered excluded and will be deleted
on the destination. Test first with
--dry-run if you are not sure
what will happen.
When checking whether a file has been modified, this is the maximum allowed time difference that a file can have and still be considered equivalent.
The default is
1ns unless this is overridden by a remote. For
example OS X only stores modification times to the nearest second so
if you are reading and writing to an OS X filing system this will be
1s by default.
This command line flag allows you to override that computed default.
Accept-Encoding: gzip. This means that rclone won’t ask
the server for compressed files automatically. Useful if you’ve set
the server to return files with
Content-Encoding: gzip but you
uploaded compressed files.
There is no need to set this in normal operation, and doing so will decrease the network transfer efficiency of rclone.
Normally rclone outputs stats and a completion message. If you set this flag it will make as little output as possible.
Retry the entire sync if it fails this many times it fails (default 3).
Some remotes can be unreliable and a few retries helps pick up the files which didn’t get transferred because of errors.
Disable retries with
Normally rclone will look at modification time and size of files to see if they are equal. If you set this flag then rclone will check only the size.
This can be useful transferring files from dropbox which have been modified by the desktop sync client which doesn’t set checksums of modification times in the same way as rclone.
Rclone will print stats at regular intervals to show its progress.
This sets the interval.
The default is
1m. Use 0 to disable.
This option allows you to specify when files on your destination are deleted when you sync folders.
Specifying the value
--delete-before will delete all files present on the
destination, but not on the source before starting the transfer
of any new or updated files.
--delete-during (default value) will delete files while checking
and uploading files. This is usually the fastest option.
--delete-after will delay deletion of files until all new/updated
files have been successfully transfered.
This sets the IO idle timeout. If a transfer has started but then becomes idle for this long it is considered broken and disconnected.
The default is
5m. Set to 0 to disable.
The number of file transfers to run in parallel. It can sometimes be useful to set this to a smaller number if the remote is giving a lot of timeouts or bigger if you have lots of bandwidth and a fast remote.
The default is to run 4 file transfers in parallel.
This forces rclone to skip any files which exist on the destination and have a modified time that is newer than the source file.
If an existing destination file has a modification time equal (within the computed modify window precision) to the source file’s, it will be updated if the sizes are different.
On remotes which don’t support mod time directly the time checked will be the uploaded time. This means that if uploading to one of these remoes, rclone will skip any files which exist on the destination and have an uploaded time that is newer than the modification time of the source file.
This can be useful when transferring to a remote which doesn’t support
mod times directly as it is more accurate than a
and faster than using
If you set this flag, rclone will become very verbose telling you about every file it considers and transfers.
Very useful for debugging.
Prints the version number
Your configuration file contains information for logging in to
your cloud services. This means that you should keep your
.rclone.conf file in a secure location.
If you are in an environment where that isn’t possible, you can add a password to your configuration. This means that you will have to enter the password every time you start rclone.
To add a password to your rclone configuration, execute
>rclone config Current remotes: e) Edit existing remote n) New remote d) Delete remote s) Set configuration password q) Quit config e/n/d/s/q>
s, Set configuration password:
e/n/d/s/q> s Your configuration is not encrypted. If you add a password, you will protect your login information to cloud services. a) Add Password q) Quit to main menu a/q> a Enter NEW configuration password: password: Confirm NEW password: password: Password set Your configuration is encrypted. c) Change Password u) Unencrypt configuration q) Quit to main menu c/u/q>
Your configuration is now encrypted, and every time you start rclone you will now be asked for the password. In the same menu you can change the password or completely remove encryption from your configuration.
There is no way to recover the configuration if you lose your password.
rclone uses nacl secretbox which in turn uses XSalsa20 and Poly1305 to encrypt and authenticate your configuration with secret-key cryptography. The password is SHA-256 hashed, which produces the key for secretbox. The hashed password is not stored.
While this provides very good security, we do not recommend storing your encrypted rclone configuration in public if it contains sensitive information, maybe except if you use a very strong password.
If it is safe in your environment, you can set the
environment variable to contain your password, in which case it will be
used for decrypting the configuration.
If you are running rclone inside a script, you might want to disable
password prompts. To do that, pass the parameter
--ask-password=false to rclone. This will make rclone fail instead
of asking for a password if
RCLONE_CONFIG_PASS doesn’t contain
a valid password.
These options are useful when developing or debugging rclone. There
are also some more remote specific options which aren’t documented
here which are used for testing. These start with remote name eg
--drive-test-option - see the docs for the remote in question.
Write CPU profile to file. This can be analysed with
go tool pprof.
Dump HTTP headers and bodies - may contain sensitive info. Can be very verbose. Useful for debugging only.
Dump the filters to the output. Useful to see exactly what include and exclude options are filtering on.
Dump HTTP headers - may contain sensitive info. Can be very verbose. Useful for debugging only.
Write memory profile to file. This can be analysed with
go tool pprof.
--no-check-certificate controls whether a client verifies the
server’s certificate chain and host name.
--no-check-certificate is true, TLS accepts any certificate
presented by the server and any host name in that certificate.
In this mode, TLS is susceptible to man-in-the-middle attacks.
This option defaults to
This should be used only for testing.
For the filtering options
See the filtering section.
rclone has 3 levels of logging,
By default rclone logs
Info to standard error and
to standard output. This means you can redirect standard output and
standard error to different places.
By default rclone will produce
Info level messages.
If you use the
-q flag, rclone will only produce
If you use the
-v flag, rclone will produce
If you use the
--log-file=FILE option, rclone will redirect
Debug messages along with standard error to FILE.
If any errors occurred during the command, rclone will set a non zero exit code. This allows scripts to detect when rclone operations have failed.