Before posting, and to avoid disappointment, please read the following:

  • This forum is not for 2BrightSparks to provide technical support. It's primarily for users to help other users. Do not expect 2BrightSparks to answer any question posted to this forum.
  • If you find a bug in any of our software, please submit a support ticket. It does not matter if you are using our freeware, a beta version or you haven't yet purchased the software. We want to know about any and all bugs so we can fix them as soon as possible. We usually need more information and details from you to reproduce bugs and that is better done via a support ticket and not this forum.

Dealing with moved/renamed files in SFTP backup

For technical support visit https://support.2brightsparks.com/
Post Reply
Jeff13
Newbie
Newbie
Posts: 2
Joined: Mon Dec 19, 2016 5:13 pm

Dealing with moved/renamed files in SFTP backup

Post by Jeff13 »

I've been reading through the forum and the instructions to try and figure this out, but I can't seem to find a proper solution. Basically I have a drive on a Windows server that I want to backup offsite to an SFTP server. The drive is HUGE (500GB of data), so I already did a local backup in-house before relocating the SFTP drive off-site, since the Internet connection on site is disgustingly slow (5mbit down, 0.7mbit up). Now I want to maintain a backup of the files from the server to the off-site SFTP. I have it set SyncBackPro to run a backup every night from the server to the SFTP. It was working, but then I ran into a huge problem. Someone moved a folder with 15GB of data into another folder. The file contents didn't change - they were just moved. Syncback then tried to copy the contents of the new folder location to the SFTP and it failed as it was taking days to complete the upload. But there is no need to upload the contents - they haven't changed, only the folder location moved. So I thought to try using Intelligent Synchronizationhronization instead of Backup, but I'm at a loss on how to configure it. The option of "Detect file renames on Source" is greyed out so I can't choose it. Nothing in the instructions explains why. Also, I don't really want to do a sync as nothing in the SFTP backup should ever be given priority over the server contents, I want the server ("left") to always be backed up to the SFTP ("right") and files to NEVER be removed or changed on the "left" as a result of this backup. But I need the software to recognize files/folders that are moved or renamed on the "left" and not re-upload them to the "right", but rather just move/rename them locally on the "right" since bandwidth and Internet speed at the server location are extremely limited. How can I properly set SyncBackPro to do this?
Swapna
2BrightSparks Staff
2BrightSparks Staff
Posts: 992
Joined: Mon Apr 13, 2015 6:22 am

Re: Dealing with moved/renamed files in SFTP backup

Post by Swapna »

Hi,

Only a Intelligent Synchronization profile in SyncBackPro supports file rename detection using the options “Detect file renames on left/right" (this feature is not available in Backup or Mirror profile types).

In order to detect file renames, SyncBackPro will compute and compare the hash value of files on profile’s Left and Right locations. However, SyncBackPro (by default) does not perform hashing when using SFTP, as very few SFTP servers implement it. Thus, the file rename detection options were disabled (or greyed out) if the Profile’s Left or Right is an SFTP server.

One possible workaround would be:

1. Switch the profile type to Backup

2. Then try to remote access into the offsite SFTP server and manually move that folder to the same location to match with the Source. Please contact your SFTP admins for further assistance, if you cannot remote access or manually move files on the SFTP server

3. After the files are moved on SFTP side, run the Backup profile to ensure that the file contents and properties on Source and Destination are identical.

Thank you.
Jeff13
Newbie
Newbie
Posts: 2
Joined: Mon Dec 19, 2016 5:13 pm

Re: Dealing with moved/renamed files in SFTP backup

Post by Jeff13 »

Unfortunately, I need the backup to determine when folders or files have moved or renamed on its own because the client tends to shuffle things randomly. I have direct access to the SFTP server as it is connected to a Lan at one of my locations. It is a Seagate D2 NAS. Would there be some way to activate the hashing feature on it? I suppose I can ask Seagate. Is that all that would be needed in order use the feature of Intelligent Synchronization to detect moved files? Is hashing something that may be available over standard FTP instead, or another protocol that NAS supports? I'm just looking for any way to back up this data that won't require a complete re-upload every time someone moves or renames a folder.
Swapna
2BrightSparks Staff
2BrightSparks Staff
Posts: 992
Joined: Mon Apr 13, 2015 6:22 am

Re: Dealing with moved/renamed files in SFTP backup

Post by Swapna »

Hi,
I have direct access to the SFTP server as it is connected to a Lan at one of my locations. It is a Seagate D2 NAS. Would there be some way to activate the hashing feature on it?
Sorry, "detect file renames/moves" feature is not available when using SFTP, and there is no way to activate this feature with SFTP.
Is hashing something that may be available over standard FTP instead, or another protocol that NAS supports?
Yes, the options to detect file renames/moves in a Intelligent Synchronization profile are available when using plain FTP or FTPS. However, these options will work only if your FTP server support the XCRC extension and file renames/moves are detected only from run-2 onwards as in run-1 there is no previous-state database.

In addition, there are some points to consider when using detect file renames/moves options:

1. As SyncBack has to compute and compare the hash values of files on both sides, the profile run can be very slow (sometimes as slow as a 'delete and re-copy') and also it is stated in the help file that it will try to find such files, so success is not guaranteed.

2. Note these options are not available for folders. Therefore, if a folder is moved (without changing the file name), then the only option available is to delete the old folder and its contents from the destination side and then copy the new folder and its files again from Source to Destination

3. If the files have been edited and re-named, it will never find them (because they fail the file-comparison, because the files no longer have the same content, as they are edited).

Please read the Help file for detailed information. With Decision-Files settings page opened, press F1 to open the contextual help.

Thank you.
Post Reply