I've been running vacuum pumps for quite a while at the track. My first experience with one was in 2005, when I added one to the 510" FE in my Mach 1. Back to back runs at the track with the belt for the pump disconnected and then connected resulted in an improvement from 10.60 to 10.45, probably about 20 HP. IIRC I was running about 14" of vacuum.
Sealing up the crankcase is the difficult part of using a vacuum pump; common leak areas are the distributor seal and dipstick. Also, on the street I have found that they are not that practical, because they tend to pull oil out of the engine at a fairly rapid rate. Then the catch can fills up and overflows, and you have oil all over the engine compartment. Ask me how I know this

Anyway, the easy solution is to run the pump at the track only, and take the belt off for street use, then install a breather on the valve cover instead of connecting the line to the pump. Works good that way...