There's a simple way to do this:
go to Security tab in Properties, Edit permissions, you'd keep System, Administrators, "your user" and remove other users
they can not access your folder without admin's password, <log in other user to check>
(You can add them back anytime)