Before posting, and to avoid disappointment, please read the following:

  • This forum is not for 2BrightSparks to provide technical support. It's primarily for users to help other users. Do not expect 2BrightSparks to answer any question posted to this forum.
  • If you find a bug in any of our software, please submit a support ticket. It does not matter if you are using our freeware, a beta version or you haven't yet purchased the software. We want to know about any and all bugs so we can fix them as soon as possible. We usually need more information and details from you to reproduce bugs and that is better done via a support ticket and not this forum.

worker threads for network (not cloud) backups

For technical support visit https://support.2brightsparks.com/
Post Reply
Statick
Newbie
Newbie
Posts: 2
Joined: Sun Dec 22, 2019 3:38 pm

worker threads for network (not cloud) backups

Post by Statick »

I intend to use SBP with cloud software Mountain Duck, which is software that mounts your chosen cloud storage (in my case Google Drive) as a network share, and encrypts/decrypts all files in realtime using Cryptomator encryption. Cryptomator is an open source encryption technology that encrypts files, filenames, and folder structures, extremely quickly and in a way that still works well with cloud storage (although in theory you could create Cryptomator vaults on any storage), so each file and folder remains separate (so changing one file means uploading one file), but all filenames and the folder structure are completely obfuscated as well as the file contents themselves. I've also seen other software that works similarly, e.g Boxcryptor does the same thing and also mounts your cloud storage as a network share

I've been testing the free trial of SBP and it works well backing up my files to this system - although SBP directly supports Google Drive and other cloud providers, because this software mounts the cloud storage as a network share I have to use a regular file system backup, which means that due to the nature of cloud storage, it becomes extremely slow when backing up small files, as there is a long pause when opening and closing each file.

I've seen in the documentation that when using the actual cloud backups in SBP (e.g. connecting directly to Google Drive instead of connecting via an encrypted network share), you can use worker threads to back up multiple files concurrently when the files are small, so that it doesn't slow to a crawl as a result of waiting for each file to open and close in turn

Is there any plan to make this feature available to regular file system backups as well? It would be an absolute game changer for me. Currently I'm seeing backup speeds of 2MB/s for large files and, well, bytes/sec for small ones, because there's no way to use worker threads. It occurs to me that although my setup is probably extremely niche, there will always be unsupported cloud providers, and mounting the cloud storage as a network share is becoming a more common option for connecting. You will also encounter similar speed issues around network shares connected over VPNs over the internet. All of these instances would be served by making this feature available for regular file system backups.

Additionally, would there be any likelihood at all of supporting Cryptomator vaults directly within SBP? This is open source cloud encryption technology, it works extremely well, and if it were supported in future versions it would allow us to use SBP to connect directly to our chosen cloud provider without using the mounted network share method, it would also bring a much friendlier and more widely supported encryption method to SBP backups in general
Swapna
Expert
Expert
Posts: 1026
Joined: Mon Apr 13, 2015 6:22 am

Re: worker threads for network (not cloud) backups

Post by Swapna »

Hi,

Worker threads (parallel uploads/downloads) is not supported with Google Drive. It is supported with Amazon S3, Microsoft Azure, Egnyte, Backblaze B2, Rackspace/OpenStack and Google Storage.

However, I will pass your feature request to the relevant department.

Thank you.
Post Reply