b2: implement large file uploading - fixes #456

This commit is contained in:
Nick Craig-Wood
2016-06-15 18:49:11 +01:00
parent 1b4370bde1
commit 2a46be8cf3
5 changed files with 426 additions and 35 deletions

View File

@@ -1,7 +1,7 @@
---
title: "B2"
description: "Backblaze B2"
date: "2015-12-29"
date: "2016-06-15"
---
<i class="fa fa-fire"></i>Backblaze B2
@@ -106,6 +106,9 @@ method to set the modification time independent of doing an upload.
The SHA1 checksums of the files are checked on upload and download and
will be used in the syncing process. You can use the `--checksum` flag.
Large files which are uploaded in chunks will store their SHA1 on the
object as `X-Bz-Info-large_file_sha1` as recommended by Backblaze.
### Versions ###
When rclone uploads a new version of a file it creates a [new version
@@ -130,8 +133,26 @@ depending on your hardware, how big the files are, how much you want
to load your computer, etc. The default of `--transfers 4` is
definitely too low for Backblaze B2 though.
### Specific options ###
Here are the command line options specific to this cloud storage
system.
#### --b2-chunk-size valuee=SIZE ####
When uploading large files chunk the file into this size. Note that
these chunks are buffered in memory. 100,000,000 Bytes is the minimim
size (default 96M).
#### --b2-upload-cutoff=SIZE ####
Cutoff for switching to chunked upload (default 4.657GiB ==
5GB). Files above this size will be uploaded in chunks of
`--b2-chunk-size`. The default value is the largest file which can be
uploaded without chunks.
### API ###
Here are [some notes I made on the backblaze
API](https://gist.github.com/ncw/166dabf352b399f1cc1c) while
integrating it with rclone which detail the changes I'd like to see.
integrating it with rclone.