excluding filter from category | WordPress.org

How can I exclude a global filter from certain categories?

I have created a universal tag filter that is filtering my product categories, however, I would like to exclude this filter from 3 of my 4 categories and I cannot seem to figure out how to do that? Any help would be greatly appreciated.