None of this makes sense to me. I've had slip joint headers that showed the black marks from exhaust exiting between the joints....never smelled anything, or heard leaks. IF there was anything to hear, it was only at full throttle, where I couldn't hear it anyway over the carb and/or exhaust note. I most definitely never smelled like exhaust from driving, or smelled it at all for that matter.
Not counting a really REALLY bad leak at the head or collector, the only way I've smelled exhaust in the car (also a Mach 1), was when my exhaust terminated at the rear axle, OR when the exhaust didn't protrude past the rear valance, where it gets trapped under the car and makes its way back into the trunk, or back to the side windows. That is definitely an issue. I just don't believe a small joint leak could produce enough fumes to smell inside, let alone cover the driver. Almost 40 years of driving my Mach, with different engines, different headers, different mufflers...or no exhaust at all, that is my experience.
Sound can be a funny thing. Certain frequencies can trigger really annoying reactions, especially as a person ages and their hearing changes. It may be the case here, where a certain frequency from those particular thin wall headers is just not playing well with your hearing. But in fairness, I've never ran REF headers, so can't say whether or not they are worse, or just different, than others.