Tao Effect Forums

Espionage => Espionage 2 => Topic started by: pmdf on September 10, 2010, 10:22:50 PM

Title: 'Endless' Network Backup
Post by: pmdf on September 10, 2010, 10:22:50 PM
I've recently been testing Espionage, and the way it manages encrypted disk images seems ideal for my requirements. However, I've encountered an issue with the built-in backup facility.  

Having set up all my encrypted folder today, I attempted to run my first backup. The backup was to a networked drive, connected via AFP.  The local machine's OS in OS X 10.6.4 and the remote (on an internal network, via gigabit ethernet) machine's OS is 10.5.8.

I left the backup running for a couple of hours, and when I came back noticed that Espionage was still attempting to backup the first folder.  I was a bit surprised, as although it's a reasonably large folder (approx 1GB) I didn't expect it to take that long. Checking the backup folder on the networked drive I noticed well over 100,000 zero kb sparse bundle files had been created and this was still increasing very quickly! I stopped the backup by restarting my machine and tried a much smaller folder, but had exactly the same issue - within minutes there were several thousand zero kb files.

As a test, I created a new folder and placed a single 300kb file inside it and tried backing it up to the network drive. Again, within a couple of minutes I had over 7,000 zero kb files in the destination folder.

Wondering whether it was the fact it was a networked drive causing the issue, I attempted to backup to a local folder instead. The backup completed successfully within seconds.

So it seems that Espionage doesn't like my network! Can you advise if there might be something I'm doing incorrectly that could fix this issue? This is pretty key for me, as the networked machine backs up to the cloud overnight, so is an important element of my backup strategy.

Thanks in advance! :)
Title: Re: 'Endless' Network Backup
Post by: pmdf on September 11, 2010, 12:22:46 PM
Just as an additional bit of info... I've just tried backing up to my networked Time Capsule, and to a USB drive connected to the Time Capsule and it works fine on both. That probably solves my problem, as this can be accessed for the overnight backup to the cloud, but I'd still be interested in why I'm getting the odd behaviour with the other networked drives.
Title: Re: 'Endless' Network Backup
Post by: greg on September 11, 2010, 06:51:57 PM
Hi pmdf,

That sounds like a very rare and odd situation you've discovered. :-)

We use the UNIX program called rsync (http://http://en.wikipedia.org/wiki/Rsync) that comes with OS X to perform Espionage's backups. This utility has been tested in all sorts of situations, so usually it's pretty good at what it does (incremental backups).

I just tested having Espionage backup to another computer that was mounted via Apple File Sharing and that worked just fine, but perhaps there are special cases were rsync has this odd behavior. Unfortunately there's not much we can do about it (as far as I'm currently aware), but thank you for the detailed description, it could come in handy later, and just serve as a pointer to others who might experience this issue.

I'm glad to hear though that you found an alternative backup route.

If I can assist in any other way let me know!
Title: Re: 'Endless' Network Backup
Post by: pmdf on September 11, 2010, 07:12:04 PM
Thanks for the reply Greg. Interestingly I also use ChronoSync every day to sync with the same network drive. I believe that uses rsync too? Strange how it should work but this doesn't.

Anyway, I have a good workaround, as described, so no harm done and I'm finding Espionage a really fantastic tool... Wish I had discovered it years ago!  I nearly overlooked it as I didn't see what advantage it could offer over me manually managing my encrypted disk images, but I'm totally sold. Now just to decide whether to buy it outright or as a part of the MacUpdate bundle.  :)
Title: Re: 'Endless' Network Backup
Post by: niella on December 23, 2010, 04:59:20 AM

I have the exact same issue, even though I've formerly been able to use Espionage backups to my Time Capsule. Backups now just execute endlessly, and I can also see the thousands of sparse files that the OP mentioned. I erased my time capsule and did a 200GB Time Machine backup afterwards without any issues subsequently, but I just cannot find a way to get the Espionage based backups to work and ALL of my critical data sits on the encrypted images... :(

There appear to be 3 rsync processes per backup, it remained there for an entire evening even though the image to backup was not all that large:
ps -ef|grep -i rsync
  501   825   224   0   0:00.02 ??         0:00.02 /usr/bin/rsync -aE --delete --delete-after /Volumes/MacData/data/src/my_image.git/.my_image.git.sparsebundle /Volumes/Data/Espionage/EspionageBackups/niella/-960678611/my_image.git
  501   826   825   0   0:00.01 ??         0:00.01 /usr/bin/rsync -aE --delete --delete-after /Volumes/MacData/data/src/my_image.git/.my_image.git.sparsebundle /Volumes/Data/Espionage/EspionageBackups/niella/-960678611/my_image.git
  501   827   826   0   5:07.54 ??         5:14.21 /usr/bin/rsync -aE --delete --delete-after /Volumes/MacData/data/src/my_image.git/.my_image.git.sparsebundle /Volumes/Data/Espionage/EspionageBackups/niella/-960678611/my_image.git

Worst of all is, it seems I cannot delete the (partial) backup images on the destination drive, since tools like ls just shows "ls: ..DS_Store.zx3EYc: No such file or directory" hundreds of times.

Must say, I'm a bit at a loss as to what to do next.
Title: Re: 'Endless' Network Backup
Post by: niella on December 23, 2010, 11:38:25 AM
In case anyone else also encounters this problem, I've found that it relates to the new Apple 7.5.2 firmware for the Time Capsule. After re-installing AirPortUtility551 (firmware 7.5.1), I am able to a) delete all those pesky temporary rsync files from the Time Capsule without having to resort to "erase" and b) use Espionage to perform backups to my Time Capsule again.

Title: Re: 'Endless' Network Backup
Post by: pmdf on December 23, 2010, 06:08:37 PM
Just to confirm that I too am again getting this problem after upgrading the Time Capsule firmware.  As using the Time Capsule was my only workaround to the exact same problem with other networked drives, that now leaves me without any solution at all for backing up over my network. :(

Edit: looks like it's a bug with AFP that Apple need to fix: http://discussions.apple.com/thread.jsp ... &tstart=75 (http://discussions.apple.com/thread.jspa?threadID=2689496&tstart=75). Mounting the network drives via SMB solves the problem.
Title: Re: 'Endless' Network Backup
Post by: greg on December 26, 2010, 11:12:22 PM
Thanks for sharing your findings and sorry I wasn't of much help on this!
Title: Re: 'Endless' Network Backup
Post by: kettysimpson on December 07, 2011, 04:57:06 PM
I have a good experience with Espionage, as described, didn't get any problem with it and I found  Espionage a really very helpful tool. Wish i would have patented it.! I checked out it carefully and I didn't find any advantage it could offer over me manually managing my encrypted disk images, but I'm was totally wrong. Now i have decided to buy it anyhow