Backblaze B2
B2 is Backblaze's cloud storage system.
Paths are specified as remote:bucket
(or remote:
for the lsd
command.) You may put subdirectories in too, e.g. remote:bucket/path/to/dir
.
Here is an example of making a b2 configuration. First run
This will guide you through an interactive setup process. To authenticateyou will either need your Account ID (a short hex number) and MasterApplication Key (a long hex number) OR an Application Key, which is therecommended method. See below for further details on generating and usingan Application Key.
This remote is called remote
and can now be used like this
See all buckets
Create a new bucket
We use BackBlaze for corporate end point backups (imp laptops/desktops), not B2-Cloudflare integration though. The way it'll work is that the B2 storage will act as the Origin from where the Cloudflare POPs will pull the data and cache. B3.3 Series 50-62.5 kVA, 40-50 kWe Prime Latest Technology And Unmatched Performance The Cummins® B3.3 series rugged engine and world class Stamford alternator powered diesel generator set Advanced in-cylinder technology to meet latest emission norms without any after-treatment device. Smart aesthetic and superior finish. The Backblaze Online Backup software creates a mirror copy of the data on the licensed computer, except for data explicitly excluded, with a running 30-day file revision history. Backblaze has introduced extended version history that, for an additional fee, will keep your data for up to 1-year or permanently.
Follow Backblaze's docs to create an Application Key with the required permission and add the applicationKeyId as the account and the Application Key itself as the key. Note that you must put the applicationKeyId as the account – you can't use the master Account ID. Using cell references, where $100 is in cell B3 and $50 is in cell B4, the excel formula required is =B4/B3. How to Backup a Windows or Mac PC to the Cloud using Backblaze.
List the contents of a bucket
Sync /home/local/directory
to the remote bucket, deleting anyexcess files in the bucket.
Application Keys
B2 supports multiple Application Keys for different access permissionto B2 Buckets.
You can use these with rclone too; you will need to use rclone version 1.43or later.
Follow Backblaze's docs to create an Application Key with the requiredpermission and add the applicationKeyId
as the account
and theApplication Key
itself as the key
.
Note that you must put the applicationKeyId as the account
– youcan't use the master Account ID. If you try then B2 will return 401errors.
--fast-list
This remote supports --fast-list
which allows you to use fewertransactions in exchange for more memory. See the rclonedocs for more details.
Modified time
The modified time is stored as metadata on the object asX-Bz-Info-src_last_modified_millis
as milliseconds since 1970-01-01in the Backblaze standard. Other tools should be able to use this asa modified time.
Modified times are used in syncing and are fully supported. Note thatif a modification time needs to be updated on an object then it willcreate a new version of the object.
Restricted filename characters
In addition to the default restricted characters setthe following characters are also replaced:
Character | Value | Replacement |
---|---|---|
0x5C | \ |
Invalid UTF-8 bytes will also be replaced,as they can't be used in JSON strings.
Note that in 2020-05 Backblaze started allowing characters in filenames. Rclone hasn't changed its encoding as this could cause syncs tore-transfer files. If you want rclone not to replace then see the--b2-encoding
flag below and remove the BackSlash
from thestring. This can be set in the config.
SHA1 checksums
The SHA1 checksums of the files are checked on upload and download andwill be used in the syncing process.
Large files (bigger than the limit in --b2-upload-cutoff
) which areuploaded in chunks will store their SHA1 on the object asX-Bz-Info-large_file_sha1
as recommended by Backblaze.
For a large file to be uploaded with an SHA1 checksum, the sourceneeds to support SHA1 checksums. The local disk supports SHA1checksums so large file transfers from local disk will have an SHA1.See the overview for exactly which remotessupport SHA1.
Sources which don't support SHA1, in particular crypt
will uploadlarge files without SHA1 checksums. This may be fixed in the future(see #1767).
Files sizes below --b2-upload-cutoff
will always have an SHA1regardless of the source.
Transfers
Backblaze recommends that you do lots of transfers simultaneously formaximum speed. In tests from my SSD equipped laptop the optimumsetting is about --transfers 32
though higher numbers may be usedfor a slight speed improvement. The optimum number for you may varydepending on your hardware, how big the files are, how much you wantto load your computer, etc. The default of --transfers 4
isdefinitely too low for Backblaze B2 though.
Note that uploading big files (bigger than 200 MB by default) will usea 96 MB RAM buffer by default. There can be at most --transfers
ofthese in use at any moment, so this sets the upper limit on the memoryused.
Versions
When rclone uploads a new version of a file it creates a new versionof it.Likewise when you delete a file, the old version will be marked hiddenand still be available. Conversely, you may opt in to a 'hard delete'of files with the --b2-hard-delete
flag which would permanently removethe file instead of hiding it.
Old versions of files, where available, are visible using the--b2-versions
flag.
NB Note that --b2-versions
does not work with crypt at themoment #1627. Using--backup-dir with rclone is the recommendedway of working around this.
If you wish to remove all the old versions then you can use therclone cleanup remote:bucket
command which will delete all the oldversions of files, leaving the current ones intact. You can alsosupply a path and only old versions under that path will be deleted,e.g. rclone cleanup remote:bucket/path/to/stuff
.
Note that cleanup
will remove partially uploaded files from the bucketif they are more than a day old.
When you purge
a bucket, the current and the old versions will bedeleted then the bucket will be deleted.
However delete
will cause the current versions of the files tobecome hidden old versions.
Here is a session showing the listing and retrieval of an oldversion followed by a cleanup
of the old versions.
Show current version and all the versions with --b2-versions
flag.
Retrieve an old version
Clean up all the old versions and show that they've gone.
Data usage
It is useful to know how many requests are sent to the server in different scenarios.
All copy commands send the following 4 requests:
The b2_list_file_names
request will be sent once for every 1k filesin the remote path, providing the checksum and modification time ofthe listed files. As of version 1.33 issue#818 causes extra requeststo be sent when using B2 with Crypt. When a copy operation does notrequire any files to be uploaded, no more requests will be sent.
Uploading files that do not require chunking, will send 2 requests perfile upload:
Uploading files requiring chunking, will send 2 requests (one each tostart and finish the upload) and another 2 requests for each chunk:
Versions
Versions can be viewed with the --b2-versions
flag. When it is setrclone will show and act on older versions of files. For example
Listing without --b2-versions
And with
Showing that the current version is unchanged but older versions canbe seen. These have the UTC date that they were uploaded to theserver to the nearest millisecond appended to them.
Note that when using --b2-versions
no file write operations arepermitted, so you can't upload files or delete them.
B2 and rclone link
Backblaze B2 Api
Rclone supports generating file share links for private B2 buckets.They can either be for a file for example:
or if run on a directory you will get:
you can then use the authorization token (the part of the url from the?Authorization=
on) on any file path under that directory. For example:
Standard Options
Here are the standard options specific to b2 (Backblaze B2).
--b2-account
Account ID or Application Key ID
- Config: account
- Env Var: RCLONE_B2_ACCOUNT
- Type: string
- Default: '
--b2-key
Application Key
- Config: key
- Env Var: RCLONE_B2_KEY
- Type: string
- Default: '
--b2-hard-delete
Permanently delete files on remote removal, otherwise hide files.
- Config: hard_delete
- Env Var: RCLONE_B2_HARD_DELETE
- Type: bool
- Default: false
Advanced Options
Here are the advanced options specific to b2 (Backblaze B2).
--b2-endpoint
Endpoint for the service.Leave blank normally.
- Config: endpoint
- Env Var: RCLONE_B2_ENDPOINT
- Type: string
- Default: '
--b2-test-mode
A flag string for X-Bz-Test-Mode header for debugging.
This is for debugging purposes only. Setting it to one of the stringsbelow will cause b2 to return specific errors:
- 'fail_some_uploads'
- 'expire_some_account_authorization_tokens'
- 'force_cap_exceeded'
These will be set in the 'X-Bz-Test-Mode' header which is documentedin the b2 integrations checklist.
- Config: test_mode
- Env Var: RCLONE_B2_TEST_MODE
- Type: string
- Default: '
--b2-versions
Include old versions in directory listings.Note that when using this no file write operations are permitted,so you can't upload files or delete them.
- Config: versions
- Env Var: RCLONE_B2_VERSIONS
- Type: bool
- Default: false
--b2-upload-cutoff
Cutoff for switching to chunked upload.
Files above this size will be uploaded in chunks of '--b2-chunk-size'.
This value should be set no larger than 4.657GiB ( 5GB).
- Config: upload_cutoff
- Env Var: RCLONE_B2_UPLOAD_CUTOFF
- Type: SizeSuffix
- Default: 200M
--b2-copy-cutoff
Cutoff for switching to multipart copy
Any files larger than this that need to be server-side copied will becopied in chunks of this size.
The minimum is 0 and the maximum is 4.6GB.
- Config: copy_cutoff
- Env Var: RCLONE_B2_COPY_CUTOFF
- Type: SizeSuffix
- Default: 4G
--b2-chunk-size
Upload chunk size. Must fit in memory.
When uploading large files, chunk the file into this size. Note thatthese chunks are buffered in memory and there might a maximum of'--transfers' chunks in progress at once. 5,000,000 Bytes is theminimum size.
- Config: chunk_size
- Env Var: RCLONE_B2_CHUNK_SIZE
- Type: SizeSuffix
- Default: 96M
--b2-disable-checksum
Disable checksums for large (> upload cutoff) files
Normally rclone will calculate the SHA1 checksum of the input beforeuploading it so it can add it to metadata on the object. This is greatfor data integrity checking but can cause long delays for large filesto start uploading.
- Config: disable_checksum
- Env Var: RCLONE_B2_DISABLE_CHECKSUM
- Type: bool
- Default: false
--b2-download-url
Custom endpoint for downloads.
Backblaze B3 For Sale
This is usually set to a Cloudflare CDN URL as Backblaze offersfree egress for data downloaded through the Cloudflare network.Rclone works with private buckets by sending an 'Authorization' header.If the custom endpoint rewrites the requests for authentication,e.g., in Cloudflare Workers, this header needs to be handled properly.Leave blank if you want to use the endpoint provided by Backblaze.
- Config: download_url
- Env Var: RCLONE_B2_DOWNLOAD_URL
- Type: string
- Default: '
--b2-download-auth-duration
Time before the authorization token will expire in s or suffix ms|s|m|h|d.
The duration before the download authorization token will expire.The minimum value is 1 second. The maximum value is one week.
- Config: download_auth_duration
- Env Var: RCLONE_B2_DOWNLOAD_AUTH_DURATION
- Type: Duration
- Default: 1w
--b2-memory-pool-flush-time
How often internal memory buffer pools will be flushed.Uploads which requires additional buffers (f.e multipart) will use memory pool for allocations.This option controls how often unused buffers will be removed from the pool.
- Config: memory_pool_flush_time
- Env Var: RCLONE_B2_MEMORY_POOL_FLUSH_TIME
- Type: Duration
- Default: 1m0s
--b2-memory-pool-use-mmap
Whether to use mmap buffers in internal memory pool.
- Config: memory_pool_use_mmap
- Env Var: RCLONE_B2_MEMORY_POOL_USE_MMAP
- Type: bool
- Default: false
--b2-encoding
This sets the encoding for the backend.
See: the encoding section in the overview for more info.
- Config: encoding
- Env Var: RCLONE_B2_ENCODING
- Type: MultiEncoder
- Default: Slash,BackSlash,Del,Ctl,InvalidUtf8,Dot
Limitations
rclone about
is not supported by the B2 backend. Backends withoutthis capability cannot determine free space for an rclone mount oruse policy mfs
(most free space) as a member of an rclone unionremote.
See List of backends that do not support rclone aboutSee rclone about
Here is a list of things I have learned about legitimate anti-aging supplements. For the time being most of this article will focus on the ins and outs of Nicotinomide Riboside because it is very confusing and there’s a lot of dangerous misinformation out there.
Nicotinomide Riboside (NR)
- Nicotinamide Riboside (NR) is a legitimate anti-aging supplement that helps maintain healthy levels of NAD, a molecule in the body that declines with age. You can think of NAD as an active form of Vitamin B3 stored and released by your liver. NR has been tested extensively in rats and in humans.
- There is no need to take the often more expensive Nicotinamide mononucleotide (NMN). NR can be considered equal for this purpose. [Source: Chris Masterjohn PhD]
- You can NOT take high doses of regular Niacin AKA Nicotinamide AKA Vitamin B3. It is NOT a cheap alternative to NR and is unsafe. The upper limit of Vitamin B3 is 35mg daily. Ignore this at your peril. [Source: Chris Masterjohn PhD]
- NR should be refrigerated, because it degrades into plain nicotinamide that is not safe to take in high doses. [Source: Dr. David Sinclair PhD]
- Resveratrol is the “accelerator pedal” that is recommended to be included with NR. [Source: Dr. David Sinclair PhD]
- Resveratrol is light sensitive and should be kept in a dark cool place, such as a basement. [Justin: Refrigerator?] When exposed to light it turns brown and is not effective. [Source: Dr. David Sinclair PhD]
- Taking 25-30mg daily of NR is safe without any special precautions. But if you exceed this dose, you should match it in equal doses with Trimethylglycine (TMG). This is because all forms of Vitamin B3 including NR consume methyl groups when their metabolites are excreted into the urine. It is theorized that if you take really high doses of NR without TMG that you may experience a significant energy crash a few weeks in, as well as other more severe health risks. [Source: Chris Masterjohn PhD]
- Thorne is a one of the most reliable brands in supplements [Source: Rhonda Patrick PhD], and they make a NR with Resveratrol supplement called ResveraCel. It seems to be the least costly option on Amazon. Remember to store it in your fridge or it will turn brown and degrade.
- It is best to take NR spread throughout the day with meals. Conversion of NR to NAD is more effective this way. This may only apply to high doses.
- Supplementing with creatine is also a good idea when taking NR. Just the standard 5g daily is fine and adequate and what is universally accepted.
- In summary, take Nicotinomide Riboside (NR) only, take it with equal amounts of TMG, and if you side with David Sinclair you can include resveratrol. Creatine is helpful too. Refrigerate your NR and Resveratrol in the dark.
Metformin
Although Metformin is considered an anti-aging drug because it affects pathways related to aging, I have not made sufficient notes to put information here at this time. I can say that this drug has been around a long time and it is generally recognized as one of the safest drugs on the market. However it is incompatible with fasting because it may prevent the creation of glucose during fasting, and you should always consult with your doctor before taking it. I mentioned it here to let you know it exists.
Backblaze B3
Rapamycin
Backblaze B3 Pro
Although Rapamycin is an anti-aging drug, this drug has not yet been tested in humans. It is currently being tested in dogs with some success. [Source: Peter Attia M.D.] I mentioned it here to let you know it exists.