Hello, you cannot remove a folder set. I know it might look bad, but in fact this makes not harm at a all.
What you should do is open that folder set, remove all folder from it, and then simply make yourself forget the password you used for it, and it will vanish :-)
There is no drawback of any kind having this folderset still there if you do not use it.
Let me know your comment or opinion.
Rgds
Zsolt