Before posting, and to avoid disappointment, please read the following:

  • This forum is not for 2BrightSparks to provide technical support. It's primarily for users to help other users. Do not expect 2BrightSparks to answer any question posted to this forum.
  • If you find a bug in any of our software, please submit a support ticket. It does not matter if you are using our freeware, a beta version or you haven't yet purchased the software. We want to know about any and all bugs so we can fix them as soon as possible. We usually need more information and details from you to reproduce bugs and that is better done via a support ticket and not this forum.

Ideas for offsite backup of tons of small files

For technical support visit https://support.2brightsparks.com/
Post Reply
Tyree
Knowledgeable
Knowledgeable
Posts: 27
Joined: Tue Aug 12, 2008 3:34 pm

Ideas for offsite backup of tons of small files

Post by Tyree »

I have been moving all my backups off-site to Google Drive for the past couple weeks. I've worked out a lot of wrinkles here and there to streamline the process. One that I can't figure out a way to tackle is uploading tons of small files.

I have just 2GB of data to upload (in this particular case), but it consists of over 225k small files. As you probably know, uploading one 2GB file takes a tiny fraction of the time it takes to upload a quarter-million files totaling 2GB.

I was hoping there was some sort of way to upload a zip file to GDrive and then expand it once it uploaded. Doesn't appear to be a way to do that. There are some extensions that allow you to unzip files on GDrive, but they use local storage to do it. So, it's basically downloading and then re-uploading them to GDrive. Obviously, that defeats the whole purpose.

Does Syncback have some method of accomplishing this? Any tips for doing an operation like this?

Thanks!
Swapna
2BrightSparks Staff
2BrightSparks Staff
Posts: 993
Joined: Mon Apr 13, 2015 6:22 am

Re: Ideas for offsite backup of tons of small files

Post by Swapna »

Hi,

SyncBackPro supports single-zip compression, where it compresses all files on Source into a big zip file (locally) and copies it to the destination. However, with cloud services, we do not recommend using single-zip compression, because the zip file uploaded to the server is recreated in each run (deleted, recreated and uploaded to the server, which is a slow process). If you wish you can test single-zip compression with Google Drive and see if it is faster than uploading multiple small files.

Thank you.
Tyree
Knowledgeable
Knowledgeable
Posts: 27
Joined: Tue Aug 12, 2008 3:34 pm

Re: Ideas for offsite backup of tons of small files

Post by Tyree »

Yeah, I knew I could do that. But, like you said, every time a file is changed the entire zip file has to be uploaded again. I think I'll be stuck doing it the long, slow way.

Thanks!
Post Reply