While there's currently no way to specify that a folder only be opened by a specific application, Espionage has an ignore list that should be used to solve these sorts of problems. I would recommend adding QuickSilver to the blacklist, you can do this from within Espionage or directly from the password prompt by clicking on the button in the lower-left:

As for Path Finder, there are several things to check:
1) Check its View Options (Command+J) and make sure that "Calculate All Sizes" is turned off for all of the parent folders of the folder in question.
2) If you're using List View, then make sure that the disclosure triangle for that folder is set to the "closed" position.
If that doesn't help, see if you can find a pattern to what sorts of scenarios cause it to appear. However, please keep in mind that for now we do not officially support Path Finder, as we've so far been focusing our development efforts on ensuring complete Finder integration. The good news though is that we do plan on improving Path Finder support in future versions, so your feedback is greatly appreciated!