Unlocking large folders in 3.6.1

[nomolo]

When I unlock my large Mail folder (65 GB) with version 3.6.1. the slider in the Espionage window is very slow to reflect the unlocked status. It remains grayed out, even though Mail has been unlocked, launched and is functioning normally (see screenshot). While its grayed out, Mail can't be locked. Clicking the "i" updates the button to its correct state. Otherwise, it remains grayed out for several minutes.   

On the other hand, I can now sometimes lock this large Mail folder without first manually quitting Mail, which hadn't been possible with 3.6.0.

[zsolt]

Hello. please allow me to test this and I will come back with my findings.

Rgds
Zsolt

[zsolt]

I tried to reproduce it, but failed, both the lock/unlock of a large folder, and locking folder while there is a document open.
From my experience with Espionage, the speed at which the folder unlocks/locks does not depend on the size of the folder, at least not in a visible way.
I do recall that sometimes when you try to mount a big disk image, like OS X install disk image or such, there is a window saying "verifying disk image", so maybe this is the time during which the folder is in this undefined state. I will check with devs if this is possible.

For the folder allowing to lock while a document is open, if I try that, I get a notification with OS error -47 which is the equivalent of "file busy" error.
Now, this is of course if there is a file open. You did not say that there was an open file, but that the Mail was running. I can hardly imagine that a running Mail does not keep any single file open, but then again, all is possible in some rare circumstances.

So what I would kindly ask you:
- re big disk image, I will check with devs about details, and you please try to reproduce it again, if you manage, then please open the console application, select system log and later "all messages" and see if there are some recent espionage related lines.
- re possibility to lock the folder while a file is open, can you please test with some other folder, other then Mail. If you do manage to reproduce it, then let me know. If you can reproduce it with mail, then while the folder is locked and Mail running, run the following command from the terminal

lsof | grep -i mail >lsof.out, and send me the lsof.out file.

Thanks
Zsolt

[nomolo]

Thanks Szolt

I have a folder action to quit Mail before locking it. So I assumed that what I observed (clicking the "Lock" button causes Mail to quit and then lock) is the intended behaviour. Isn't that right? 

(I have a similar folder action with BusyCal. Just a single folder action: Quit BusyCal.  This allows me to quit and then lock BusyCal while its running, which it does successfully.)

More often, when I try and lock Mail while it is running, Mail just quits but fails to lock, presumably because other files associated with Mail are still open. I've just tried it again and this is the output:

05/10/2014 15:14:18.126 Espionage[393]: ERROR (+[DiskMaster unmountDiskForFolderFace:mountpoint:forcefully:updateUI:error:]:533): failed because: unlockStatus != noErr
05/10/2014 15:14:18.126 Espionage[393]: ERROR (-[LockFolderTask go:]:84): failed to unmount '/Users/mf/Library/Mail': Error Domain=NSOSStatusErrorDomain Code=-47 "The operation couldn't be completed. (OSStatus error -47.)" (fBsyErr: File is busy (delete))
05/10/2014 15:14:18.126 Espionage[393]: INFO: Posted critical message: Failed to lock 'Mail': The operation couldn't be completed. (OSStatus error -47.)


I will send you the requested console outputs when I reproduce the phenomenon!

[nomolo]

I have now locked Mail again while it was running with the Inbox open. I have also observed the greyed out button after unlocking Mail. I repeated each 3 times but nothing related to Espionage has come up in the console.

[zsolt]

Hello, the mail is a bit tough to play with as it is not a simple application i.e. quite often it will not quit right away when you try to quit it.
And yes, the folder action as you set it, has the purpose to quit it before locking it.

So let me just summarise it quickly:

- the fail to lock is explainable, as Mail will often fail to quit right away, even if you try to quit it by hand. This is because it tries to close the connections first and clear up the caches before disconnecting from Mail servers. And sometimes, even after it quit, some files remain open, this you can check with the lsof command I sent you before. There is nothing much we can do here, except implement some kind of retry and inform user about this, I might file this as feature request if you are interested.

- the grayed out button happens only when you unlock mail (which is again handled through folder action - unlock the folder, launch mail), correct?
If so, I would like to see if you get the same behaviour if we get rid of the folder action.
Pls remove it, then unlock the folder first, and see if you will (ever) get this grayed out button.
You do not even need to launch mail after that. If we test in this way, it is simply a matter of unlocking a big folder.

Please let me know how it went.

Thanks
Zsolt

[nomolo]

With the folder actions in place, the greyed out button changes to white 55 seconds after clicking 'Unlock'

Without the folder actions (ie deleting 'Open Mail' and 'Unlock Folder com.apple.mail') the greyed out button changes to white 4 minutes after clicking 'Unlock'

[nomolo]

Zsolt

Sorry the above figures are the wrong way round!  Its 4 minutes to unlock with the folder actions; 55 seconds without them!

[greg]

I've created an issue for this in our internal issue system and assigned it a high priority (for next release).

[nomolo]

Thanks!

[zsolt]

Hello, as Greg mentioned, we are testing this internally, trying to reproduce it.
But in the mean time, IF you wish to spend some more time helping us out, then here is what you can do:

- you do mention version 3.6 at the beginning of your post. The problem with grayed out button, did it happen on 3.6 too? Do you have the same folder setup now and when you had version 3.6 running?
- is this all on Yosemite or on Maverick
- can you start up task manager, from the view menu, select All processes, and in the search window of the activity monitor enter "hdi" without quotes.
See if you have any hdiutil process running, probably not. Now keep the window visible, and unlock the problematic folder, there should one hdiutil process appear. In about 5 seconds after the folder is unlocked, it should go away. Let me know at which point did it go away, 5 seconds after you unlocked the folder, or 5 seconds after the button got out from the grayed state, or at some other moment.

Thanks
Zsolt

[nomolo]

I'm using Mavericks, 10.9.5

I'm not aware of having seen this problem with 3.6.0. If there was a delay, it would have been very short and I would have assumed it was normal for a large file. I saw nothing like the delay of several minutes I'm getting with 3.6.1

I have added some new encrypted folders since 3.6.0 but the Mail folder which is causing the problem has not changed, it was encrypted under 3.6.0 as well as 3.6.1

Running the procedure you suggested: hdiutil pops us then disappears about 5 seconds after Mail unlocks. It doesn't hang about until the button goes from gray to white.

Let me know if I can run any other kind of test that might help.

[zsolt]

Thanks a lot, now we know in which direction to test.
I will let you know if we find something.

Rgds
Zsolt

[greg]

Dear nomolo,

I think we're going to need a few days, and then we'll have a build ready for you to test and verify that the problem has been fixed. Sorry about this, just right now there are too many things going on because of Yosemite (for example, Ernesto, the other dev, needs to update his apps for Yosemite) and other urgent items.

In the meantime, is there some type of workaround that you can use? Perhaps temporarily removing the app actions might help?

Kind regards,
Greg

[nomolo]

Thanks Greg - no problem.