johanels/rclone
1
0
Fork 0
mirror of https://github.com/rclone/rclone.git synced 2025-12-11 22:14:05 +01:00
Code Issues Actions 4 Packages Projects Releases Wiki Activity

Version v1.68.0

Browse Source
This commit is contained in:
Nick Craig-Wood
2024-09-08 16:21:56 +01:00
parent 3f7651291b
commit 49f69196c2
105 changed files with 13735 additions and 4942 deletions
Download Patch File Download Diff File Expand all files Collapse all files

46
docs/content/commands/rclone_serve_docker.md
View File

@@ -281,6 +281,11 @@ These flags control the chunking:
--vfs-read-chunk-size SizeSuffix Read the source objects in chunks (default 128M)
--vfs-read-chunk-size-limit SizeSuffix Max chunk doubling size (default off)
--vfs-read-chunk-streams int The number of parallel streams to read at once
The chunking behaves differently depending on the `--vfs-read-chunk-streams` parameter.
### `--vfs-read-chunk-streams` == 0
Rclone will start reading a chunk of size `--vfs-read-chunk-size`,
and then double the size for each read. When `--vfs-read-chunk-size-limit` is
@@ -296,6 +301,30 @@ When `--vfs-read-chunk-size-limit 500M` is specified, the result would be
Setting `--vfs-read-chunk-size` to `0` or "off" disables chunked reading.
The chunks will not be buffered in memory.
### `--vfs-read-chunk-streams` > 0
Rclone reads `--vfs-read-chunk-streams` chunks of size
`--vfs-read-chunk-size` concurrently. The size for each read will stay
constant.
This improves performance performance massively on high latency links
or very high bandwidth links to high performance object stores.
Some experimentation will be needed to find the optimum values of
`--vfs-read-chunk-size` and `--vfs-read-chunk-streams` as these will
depend on the backend in use and the latency to the backend.
For high performance object stores (eg AWS S3) a reasonable place to
start might be `--vfs-read-chunk-streams 16` and
`--vfs-read-chunk-size 4M`. In testing with AWS S3 the performance
scaled roughly as the `--vfs-read-chunk-streams` setting.
Similar settings should work for high latency links, but depending on
the latency they may need more `--vfs-read-chunk-streams` in order to
get the throughput.
## VFS Performance
These flags may be used to enable/disable features of the VFS for
@@ -424,9 +453,9 @@ rclone serve docker [flags]
--default-permissions Makes kernel enforce access control based on the file mode (not supported on Windows)
--devname string Set the device name - default is remote:path
--dir-cache-time Duration Time to cache directory entries for (default 5m0s)
--dir-perms FileMode Directory permissions (default 0777)
--dir-perms FileMode Directory permissions (default 777)
--direct-io Use Direct IO, disables caching of data
--file-perms FileMode File permissions (default 0666)
--file-perms FileMode File permissions (default 666)
--forget-state Skip restoring previous state
--fuse-flag stringArray Flags or arguments to be passed direct to libfuse/WinFsp (repeat if required)
--gid uint32 Override the gid field set by the filesystem (not supported on Windows) (default 1000)
@@ -446,7 +475,7 @@ rclone serve docker [flags]
--socket-addr string Address <host:port> or absolute path (default: /run/docker/plugins/rclone.sock)
--socket-gid int GID for unix socket (default: current process GID) (default 1000)
--uid uint32 Override the uid field set by the filesystem (not supported on Windows) (default 1000)
--umask int Override the permission bits set by the filesystem (not supported on Windows) (default 2)
--umask FileMode Override the permission bits set by the filesystem (not supported on Windows) (default 002)
--vfs-block-norm-dupes If duplicate filenames exist in the same directory (after normalization), log an error and hide the duplicates (may have a performance cost)
--vfs-cache-max-age Duration Max time since last access of objects in the cache (default 1h0m0s)
--vfs-cache-max-size SizeSuffix Max total size of objects in the cache (default off)
@@ -459,6 +488,7 @@ rclone serve docker [flags]
--vfs-read-ahead SizeSuffix Extra read ahead over --buffer-size when using cache-mode full
--vfs-read-chunk-size SizeSuffix Read the source objects in chunks (default 128Mi)
--vfs-read-chunk-size-limit SizeSuffix If greater than --vfs-read-chunk-size, double the chunk size after each chunk read, until the limit is reached ('off' is unlimited) (default off)
--vfs-read-chunk-streams int The number of parallel streams to read at once
--vfs-read-wait Duration Time to wait for in-sequence read before seeking (default 20ms)
--vfs-refresh Refreshes the directory cache recursively in the background on start
--vfs-used-is-size rclone size Use the rclone size algorithm for Used size
@@ -468,10 +498,12 @@ rclone serve docker [flags]
--write-back-cache Makes kernel buffer writes before sending them to rclone (without this, writethrough caching is used) (not supported on Windows)
```
Options shared with other commands are described next.
See the [global flags page](/flags/) for global options not listed here.
## Filter Options
### Filter Options
Flags for filtering directory listings.
Flags for filtering directory listings
```
--delete-excluded Delete files on dest excluded from sync
@@ -498,9 +530,7 @@ Flags for filtering directory listings.
--min-size SizeSuffix Only transfer files bigger than this in KiB or suffix B|K|M|G|T|P (default off)
```
See the [global flags page](/flags/) for global options not listed here.
# SEE ALSO
## See Also
* [rclone serve](/commands/rclone_serve/) - Serve a remote over a protocol.