How to exclude files directories from quick file search, but not project folder view?

I have some build directories I occasionally like to view, but don't want them to show up when I do class or file searches using the quick search shortcuts. I know I can exclude the directories in the project structure, but then it also removed them from the right project directory sidebar. Any tips?