Sounds to me like you have something sticking inside the booster. Personally, I think 11-12 inches of vacuum should work ok unless your creeping in traffic. People tend to forget that under any form of deceleration, your vacuum is going to be high, so even marginal vacuum from a big cam should work fine under most road driving conditions. My Mach only has about 6-7 and it works fine with the stock booster unless I'm not using the throttle. In cases where I'm just creeping along in slow traffic, or areas like parking lots or my driveway, where I'm not using the throttle to accelerate and slow down, I can get about 3-4 good stops before the vacuum is gone and the pedal gets hard. I replaced my booster when I restored the car though. An old booster with some vacuum leaks would likely not work as well.
My '70 F350 had great power brakes when I bought it. It sat for about 4-5 years until I got around to changing the engine and getting it road worthy. The first time I went to drive it, I had no power brakes. I wasn't happy about having to buy a new booster and was looking around to find a replacement. I drove the truck a couple more times, hitting the brakes fairly hard at low speeds, when suddenly something inside free'd up and they started working great again. Now it sits inside and gets driven fairly regularly and I have not had a problem since. Been 2 years that way now. Yours sounds similar, like something inside is sticking until enough pressure is applied, then it free's up.
As a cam comparison, the 238/248 114 lsa cam I've used before had similar vacuum as what you have now, so I don't think your vacuum is out of line for that cam. My brakes always worked perfectly with that level of vacuum, but, different cars, different brake system, so it's hard to compare on that level.
