Before posting, and to avoid disappointment, please read the following:

  • This forum is not for 2BrightSparks to provide technical support. It's primarily for users to help other users. Do not expect 2BrightSparks to answer any question posted to this forum.
  • If you find a bug in any of our software, please submit a support ticket. It does not matter if you are using our freeware, a beta version or you haven't yet purchased the software. We want to know about any and all bugs so we can fix them as soon as possible. We usually need more information and details from you to reproduce bugs and that is better done via a support ticket and not this forum.

Critical feedback

For technical support visit https://support.2brightsparks.com/
Post Reply
Timur Born
Enthusiastic
Enthusiastic
Posts: 17
Joined: Fri Jan 01, 2016 11:25 am

Critical feedback

Post by Timur Born »

Hello everyone!

I am currently testing SyncBackPro and overall like its exhaustive options and features. Still I stumbled over several caveats that keep me from buying this tool, so I'd like to give some critical feedback on what I do not like.

- SBP uses only a single network connection to Dropbox, while the original Dropbox client uses between 6-12 connections. As a result SBP is considerably slower with many small files and still slower (regular transfer speed drops) with single large files.

- The ´cloud option "Retrieve a list of all the files and folders then filter" is unbearably slow, especially with many files. SBP seems to open several dozen handles to its database file, with the majority of these using smaller than 4 kb transfers. Neither my CPU, RAM or fast SSD are bottlenecking SBP, still it takes ages. The sum of all SBP file handles transfers maybe 3 mb/s, plus another 3 mb/s for the NTFS log, that's on a drive that can do several hundred mb/s even for high threaded randomly accessed files. My suspicion is that the file transfer algorithms used for writing to the database are at fault. Maybe they circumvent the Windows cache? Maybe writes should be bundled into single large ones instead of several dozens very small ones?

- Delta API seems to behave odd, maybe in combination with the option listed above?! I transferred a directory of over 11000 small files to my Dropbox storage via the original Dropbox client. Then I asked SBP to sync the directory again. As a result SBP tried to transfer all files again, even though no changes had been made to the files. Maybe I missed some options I should have set?!

- The licensing for home users seems fair, but the license for professional users seems a bit odd. Syncing begins with two (!) computers, still you are only getting a single computer license as professional user. So you need to buy two licences to make full use of the program, as in syncing two (!) computers (like 1 desktop + 1 laptop via some cloud storage in between).

These are the main culprits I encountered during my test.
Kostas
2BrightSparks Staff
2BrightSparks Staff
Posts: 421
Joined: Thu Sep 18, 2014 2:08 am

Re: Critical feedback

Post by Kostas »

Hi,

Regarding Dropbox, SyncBackPro is using the REST API provided by Dropbox and by doing so it has to abide by some rules when using it. These kind of APIs impose some limitations to 3rd party applications that their own sync clients don't have to (e.g. concurrent connections, max. upload size, etc). Actually they could be using a totally different set of private API for this matter and do as they please. 8)

Now, before getting to the the Delta API you'll need to understand how SyncBackPro keeps track of changes when it comes to cloud storage. Many cloud services (including Dropbox) do not allow the last modification date & time of a file to be changed (which SyncBackPro uses by default to detect the differences on both sides). This is a limitation of the cloud storage API and not SyncBackPro's. To get around such limitations, SyncBackPro builds a local database to store the files' details which it cannot store on the cloud service. When the profile runs again the next time it will use the cloud database to identify the file differences at your Source and Destination.

Having said that, it's easy to understand why the Delta didn't help much in your case. When the files are uploaded on the cloud storage via the website or their native sync tool then the metadata for those uploaded files will not be available in SyncBackPro's database. Thus, SyncBackPro assumes the files at your Source and Destination are different and proposes to recopy again to your Destination. It then, builds the database for the recopied files and also for the new/changed files uploaded to your Destination (during the first profile run). But, in the subsequent runs it will not propose to recopy the files again as the cloud database contains the details of the files uploaded to the server.

But, if you know the files are identical on both sides (that means no changes are made to the uploaded files since the last run) then you can run the profile and when the Differences Window is displayed, mass select all the identical files, right click and choose 'Use details from Source" to copy the properties from one side to another (note that this will not copy the file contents, it will only copy the meta data in the local database).

As for the licensing, if you are syncing two computers within the same local network then you only need the program to be running on one of them and being able to access the second computer via UNC path, or could use SyncBack Touch (which is free to use for a max of two connections) to help with accessing the remote computer. On the other hand syncing 2 computers via a cloud storage would need both of them having SyncBackPro installed since both need to connect to the cloud storage.

Thanks,
Kostas
[2bs]
Timur Born
Enthusiastic
Enthusiastic
Posts: 17
Joined: Fri Jan 01, 2016 11:25 am

Re: Critical feedback

Post by Timur Born »

Thanks for the reply and for explaining the delta database mechanics. The "Use Details from Source" seems to be the right option then.

Concerning your database file, I wonder why write access is so incredibly slow when the "Retrieve a list" option is used? With my directory of over 11000 small files we are talking about well over half an hour before the differences window comes up, compared to just a few seconds without using that option. It really seems to come down to SBP using so many concurrent handles to its database file that all use very small writes (many less than 2k and smaller). Pure speculation on my part, of course, but in practice I sit in front of a mostly idle PC (CPU, RAM, network, SSD) and still have to wait.

Too bad about the API restricting you to only a single connection while the Dropbox client uses anywhere between 6-12 concurrent connections (and corresponding threads). It really makes a huge difference when several hundreds or even thousands of files are worked with, but even with single large files the single connection is slower (from my location). Overall I like the Dropbox client over its competition, but SBP's own Dropbox API connection means that no second local copy is needed when encrypted compression is used for backups. Without compression there are other (less convenient) tricks available, like using Symlinks or Hardlinks.
Kostas
2BrightSparks Staff
2BrightSparks Staff
Posts: 421
Joined: Thu Sep 18, 2014 2:08 am

Re: Critical feedback

Post by Kostas »

The slow speed you are experiencing is due to the download of the folder/file structure from the cloud storage and not related to the local DB. I quote the text from the help file regarding this option:
Retrieve a list of all the files and folders then filter (faster in most situations): There are two methods for SyncBackPro to request a list of all the files and folders from the cloud storage server. It can either request the lists folder by folder (which is how it does it with local drives, FTP servers, etc.) or it can ask the server for a list of all the files and folders in one go (the default). One method may be faster than the other depending on how many of the files and folders on the cloud service are going to be filtered out by your profiles. If lots of the files are going to be ignored (due to filter and file & folder selections) then the profile may run faster if this option is switched off. If your cloud storage system has tens of thousands of files on it then you may need to disable this option as SyncBackPro may use a large amount of CPU time retrieving the list.
Thanks,
Kostas
[2bs]
Timur Born
Enthusiastic
Enthusiastic
Posts: 17
Joined: Fri Jan 01, 2016 11:25 am

Re: Critical feedback

Post by Timur Born »

Sorry, but I don't concur. The delays happen when the whole list of files already finished downloading. Furthermore Syncback's CPU load is only about 50% on a *single* CPU core (out of 8 logical cores). So neither the network nor the CPU are the bottlenecks. The culprit seems to be Syncback writing to the database file of the corresponding job, with its dozens and dozens of concurrent small writes (some down to 176 bytes only).

Curiously I just tried the same on my laptop (Win 8.1), because the evaluation period on the desktop PC (Win 7) is over. The laptop hardware is slower in every aspect (CPU, RAM, WLAN, SSD), but still SBP manages to write at 6 mb/s there instead of just 3 mb/s on the (better equipped) desktop. So maybe the laptop SSD get along better with this very specific access pattern. I don't know any other software on my systems that open such a huge number of concurrent file handles, though.

Fortunately the option can be disabled, once it's found to be the source of the delays.
Timur Born
Enthusiastic
Enthusiastic
Posts: 17
Joined: Fri Jan 01, 2016 11:25 am

Re: Critical feedback

Post by Timur Born »

And just to make sure: The limitation to a single connection is a limitation of the Dropbox API, not a deliberate choice of SBP not to use more concurrent connections?
Kostas
2BrightSparks Staff
2BrightSparks Staff
Posts: 421
Joined: Thu Sep 18, 2014 2:08 am

Re: Critical feedback

Post by Kostas »

Timur Born wrote:And just to make sure: The limitation to a single connection is a limitation of the Dropbox API, not a deliberate choice of SBP not to use more concurrent connections?
The limitation is not placed by the Dropbox API per se but during our tests we encountered erratic behaviour (heavy throttling and dropped connections) when trying to do so.

Thanks,
Kostas
[2bs]
Timur Born
Enthusiastic
Enthusiastic
Posts: 17
Joined: Fri Jan 01, 2016 11:25 am

Re: Critical feedback

Post by Timur Born »

Since I am still looking for a good solution a customer and myself I am getting back to this. Currently I am testing SBP vs. Arq and other solutions
Kostas wrote:The limitation is not placed by the Dropbox API per se but during our tests we encountered erratic behaviour (heavy throttling and dropped connections) when trying to do so.
Arq seems to be using two concurrent connections. While this is less than what the Dropbox app uses, it does at least increase the 0% throughput phases of SBP to about 50% with large files. No erratic behaviour here, so maybe you could give this another try in form of an option that allows me to set the no. of connections myself with a default of 1?!
Having said that, it's easy to understand why the Delta didn't help much in your case. When the files are uploaded on the cloud storage via the website or their native sync tool then the metadata for those uploaded files will not be available in SyncBackPro's database. Thus, SyncBackPro assumes the files at your Source and Destination are different and proposes to recopy again to your Destination. It then, builds the database for the recopied files and also for the new/changed files uploaded to your Destination (during the first profile run). But, in the subsequent runs it will not propose to recopy the files again as the cloud database contains the details of the files uploaded to the server.
It should be possible to make SBP scan the files on both sides (and then notice that the files are one and the same. The whole reason for me uploading the initial set of files via web interface or Dropbox' own utility is that SBP uploads them so much slower. This does not seem like such an unusual scenario, especially since some cloud services still offer customers to send in a seed via harddrive. Since the Dropbox utility is able to deal with this I would expect the API enabling third party software to deal with it as well?!

Unfortunately I encountered another culprit: Using compression (and encryption) only seems to utilize a single CPU core (out of 8 logical cores) with SBP. Obviously this slows down a large backup job considerably compared to solutions to fully utilize the CPU during compression.
Timur Born
Enthusiastic
Enthusiastic
Posts: 17
Joined: Fri Jan 01, 2016 11:25 am

Re: Critical feedback

Post by Timur Born »

I had another look at SBP, but unfortunately it still only uses a single upload thread for Dropbox and Onedrive connections. As a result backing up many small files takes a lot longer compared to using the respective cloud clients.

On a side note: The "Onedrive (Business)" option only seems to work with US servers of Onedrive, not with (local) german ones. The german service is rather new, but the preferred method of using Office 365 / Onedrive for german businesses.

"Application with identifier 'a178fae9-3f08-4e78-98c3-859c212c4c4a' was not found in the directory" <domainname>

"Request Id: 36cbaf96-d3dd-4da7-b73b-2deb6d780200
Correlation Id: 5cc495c3-098a-4538-b85d-095a1149df01
Timestamp: 2018-06-27T13:28:43Z
Message: AADSTS70001: Application with identifier 'a178fae9-3f08-4e78-98c3-859c212c4c4a' was not found in the directory" <domainname>
Timur Born
Enthusiastic
Enthusiastic
Posts: 17
Joined: Fri Jan 01, 2016 11:25 am

Re: Critical feedback

Post by Timur Born »

If you consider implementing access to Office 365 / Onedrive Germany in the future, this is the login site:

https://login.microsoftonline.de
Timur Born
Enthusiastic
Enthusiastic
Posts: 17
Joined: Fri Jan 01, 2016 11:25 am

Re: Critical feedback

Post by Timur Born »

I found out that I can login to Onedrive Business Germany via Sharepoint if the application allows me to enter the correct URL (//born-my.sharepoint.de/personal/<email_adress>/documents). For example, this works perfectly well with X-Change Editor (PDF Editor).

Unfortunately SyncBackPro does not allow me to enter an URL and the authorization page seems to be the same global/US one that comes up when OneDrive Business is chosen. So again I cannot login using my German credentials.

If you would include the option to enter a Sharepoint URL then I might be able to use SyncBackPro for backup to my OneDrive Germany account.

Thanks and regards.
Timur Born
Enthusiastic
Enthusiastic
Posts: 17
Joined: Fri Jan 01, 2016 11:25 am

Re: Critical feedback

Post by Timur Born »

Any news on the compression vs. core count front?

And is there any chance to get access to Onedrive Business / Sharepoint Germany (like introducing an URL option)?
Timur Born
Enthusiastic
Enthusiastic
Posts: 17
Joined: Fri Jan 01, 2016 11:25 am

Re: Critical feedback

Post by Timur Born »

Since no answers are given I assume that no improvements have been made in the corresponding areas. Thanks anyway.
Fender
Newbie
Newbie
Posts: 2
Joined: Wed Feb 26, 2020 6:14 pm

Re: Critical feedback

Post by Fender »

Kostas wrote:
Mon Feb 01, 2016 8:11 am
Regarding Dropbox, SyncBackPro is using the REST API provided by Dropbox and by doing so it has to abide by some rules when using it. These kind of APIs impose some limitations to 3rd party applications that their own sync clients don't have to (e.g. concurrent connections, max. upload size, etc). Actually they could be using a totally different set of private API for this matter and do as they please.
This is an old thread and OneDrive is mentioned in this thread as well regarding upload speeds.

When using SBP 9.2.39.0 uploading speed to OneDrive is really slow even now year 2020. Can this really only be explained by the API?

If i use CloudBerry Explorer 3.1.0.17 the upload speeds are around 100 Mbps and files are uploaded using several threads. Since MSP 360 are using the API provided by Microsoft as well one would expect the same slow upload speeds, so what´s the difference?

MSP offers better syncing possibilities than CloudBerry Explorer 3.1.0.17 but really don´t understand why the upload speed throttles all the time with SBP while CloudBerry Explorer 3.1.0.17 uses all the aviable bandwidth making uploads fast. For the moment i am using a trial version of SBP.
Post Reply