The vacuum was hooked to the ported vacuum on an Edelbrock carburetor.
I started off with 10 degrees BTDC, and advanced it from there.
Huh. Wouldn't have expected the vacuum advance to cause problems like that. If the diaphragm had a hole in it, the carb could have been sucking a little air in through the hose. Shouldn't be at idle, but if the blades were open a little too far or at fast idle, maybe. But figure at worst, it would just be a little lean. In that scenario, the timing wouldn't be affected with or without the hose because the advance wouldn't be working, just the A/F ratio would change.
If the diaphragm was okay but dialed in wrong so it was giving way more advance than it should be, maybe that could cause it? Still, hooked to ported vacuum, there shouldn't be much of a signal at idle.
I would investigate a bit more. Take the pencil out of the hose with it running and put your finger over it. Strong suction at idle is probably not ideal. If you have one of those little MityVac hand pumps, you could take the distributor cap off and hook the pump up to the advance and watch the operation, make sure it's not hanging up on something, etc. That would also show if it had a leak and wouldn't hold vacuum.
Pat