johanels/rclone
1
0
Fork 0
mirror of https://github.com/rclone/rclone.git synced 2025-12-11 22:14:05 +01:00
Code Issues Actions 4 Packages Projects Releases Wiki Activity

Version v1.68.0

Browse Source
This commit is contained in:
Nick Craig-Wood
2024-09-08 16:21:56 +01:00
parent 3f7651291b
commit 49f69196c2
105 changed files with 13735 additions and 4942 deletions
Download Patch File Download Diff File Expand all files Collapse all files

63
docs/content/commands/rclone_serve_http.md
View File

@@ -37,6 +37,7 @@ or just by using an absolute path name. Note that unix sockets bypass the
authentication - this is expected to be done with file system permissions.
`--addr` may be repeated to listen on multiple IPs/ports/sockets.
Socket activation, described further below, can also be used to accomplish the same.
`--server-read-timeout` and `--server-write-timeout` can be used to
control the timeouts on the server. Note that this is the total time
@@ -69,6 +70,20 @@ certificate authority certificate.
values are "tls1.0", "tls1.1", "tls1.2" and "tls1.3" (default
"tls1.0").
## Socket activation
Instead of the listening addresses specified above, rclone will listen to all
FDs passed by the service manager, if any (and ignore any arguments passed by --addr`).
This allows rclone to be a socket-activated service.
It can be configured with .socket and .service unit files as described in
https://www.freedesktop.org/software/systemd/man/latest/systemd.socket.html
Socket activation can be tested ad-hoc with the `systemd-socket-activate`command
systemd-socket-activate -l 8000 -- rclone serve
This will socket-activate rclone on the first connection to port 8000 over TCP.
### Template
`--template` allows a user to specify a custom markup template for HTTP
@@ -364,6 +379,11 @@ These flags control the chunking:
--vfs-read-chunk-size SizeSuffix Read the source objects in chunks (default 128M)
--vfs-read-chunk-size-limit SizeSuffix Max chunk doubling size (default off)
--vfs-read-chunk-streams int The number of parallel streams to read at once
The chunking behaves differently depending on the `--vfs-read-chunk-streams` parameter.
### `--vfs-read-chunk-streams` == 0
Rclone will start reading a chunk of size `--vfs-read-chunk-size`,
and then double the size for each read. When `--vfs-read-chunk-size-limit` is
@@ -379,6 +399,30 @@ When `--vfs-read-chunk-size-limit 500M` is specified, the result would be
Setting `--vfs-read-chunk-size` to `0` or "off" disables chunked reading.
The chunks will not be buffered in memory.
### `--vfs-read-chunk-streams` > 0
Rclone reads `--vfs-read-chunk-streams` chunks of size
`--vfs-read-chunk-size` concurrently. The size for each read will stay
constant.
This improves performance performance massively on high latency links
or very high bandwidth links to high performance object stores.
Some experimentation will be needed to find the optimum values of
`--vfs-read-chunk-size` and `--vfs-read-chunk-streams` as these will
depend on the backend in use and the latency to the backend.
For high performance object stores (eg AWS S3) a reasonable place to
start might be `--vfs-read-chunk-streams 16` and
`--vfs-read-chunk-size 4M`. In testing with AWS S3 the performance
scaled roughly as the `--vfs-read-chunk-streams` setting.
Similar settings should work for high latency links, but depending on
the latency they may need more `--vfs-read-chunk-streams` in order to
get the throughput.
## VFS Performance
These flags may be used to enable/disable features of the VFS for
@@ -575,15 +619,15 @@ rclone serve http remote:path [flags]
## Options
```
--addr stringArray IPaddress:Port or :Port to bind server to (default [127.0.0.1:8080])
--addr stringArray IPaddress:Port, :Port or [unix://]/path/to/socket to bind server to (default [127.0.0.1:8080])
--allow-origin string Origin which cross-domain request (CORS) can be executed from
--auth-proxy string A program to use to create the backend from the auth
--baseurl string Prefix for URLs - leave blank for root
--cert string TLS PEM key (concatenation of certificate and CA certificate)
--client-ca string Client certificate authority to verify clients with
--dir-cache-time Duration Time to cache directory entries for (default 5m0s)
--dir-perms FileMode Directory permissions (default 0777)
--file-perms FileMode File permissions (default 0666)
--dir-perms FileMode Directory permissions (default 777)
--file-perms FileMode File permissions (default 666)
--gid uint32 Override the gid field set by the filesystem (not supported on Windows) (default 1000)
-h, --help help for http
--htpasswd string A htpasswd file - if not provided no authentication is done
@@ -602,7 +646,7 @@ rclone serve http remote:path [flags]
--server-write-timeout Duration Timeout for server writing data (default 1h0m0s)
--template string User-specified template
--uid uint32 Override the uid field set by the filesystem (not supported on Windows) (default 1000)
--umask int Override the permission bits set by the filesystem (not supported on Windows) (default 2)
--umask FileMode Override the permission bits set by the filesystem (not supported on Windows) (default 002)
--user string User name for authentication
--vfs-block-norm-dupes If duplicate filenames exist in the same directory (after normalization), log an error and hide the duplicates (may have a performance cost)
--vfs-cache-max-age Duration Max time since last access of objects in the cache (default 1h0m0s)
@@ -616,6 +660,7 @@ rclone serve http remote:path [flags]
--vfs-read-ahead SizeSuffix Extra read ahead over --buffer-size when using cache-mode full
--vfs-read-chunk-size SizeSuffix Read the source objects in chunks (default 128Mi)
--vfs-read-chunk-size-limit SizeSuffix If greater than --vfs-read-chunk-size, double the chunk size after each chunk read, until the limit is reached ('off' is unlimited) (default off)
--vfs-read-chunk-streams int The number of parallel streams to read at once
--vfs-read-wait Duration Time to wait for in-sequence read before seeking (default 20ms)
--vfs-refresh Refreshes the directory cache recursively in the background on start
--vfs-used-is-size rclone size Use the rclone size algorithm for Used size
@@ -623,10 +668,12 @@ rclone serve http remote:path [flags]
--vfs-write-wait Duration Time to wait for in-sequence write before giving error (default 1s)
```
Options shared with other commands are described next.
See the [global flags page](/flags/) for global options not listed here.
## Filter Options
### Filter Options
Flags for filtering directory listings.
Flags for filtering directory listings
```
--delete-excluded Delete files on dest excluded from sync
@@ -653,9 +700,7 @@ Flags for filtering directory listings.
--min-size SizeSuffix Only transfer files bigger than this in KiB or suffix B|K|M|G|T|P (default off)
```
See the [global flags page](/flags/) for global options not listed here.
# SEE ALSO
## See Also
* [rclone serve](/commands/rclone_serve/) - Serve a remote over a protocol.