You definitely want to keep your activated carbon scent-absorbing hunting clothing and traditional hunting clothing separate.
Activated charcoal will absorb (technically adsorb) odors and once their "filter" is filled with odor molecules, you need to reactivate it. So, the idea is to store it (and wear it) as far from foreign odors as possible to keep the filter from filling up rapidly.
Ideally, you store both your undergarments and your outerwear in separate, scent-proof containers. Another key point is how you wash them. The perfumes found in most detergents and fabric softeners may have tainted your family washer and dryer. While it seems extreme, it is also ideal to wash and dry your hunting garments in separate machines from those that receive regular use with perfumed products.