Mark, I bored mine from 4.23 to 4.375, so I felt I was getting close to the edge with smaller gaskets. I like to make sure I clearly cover the fire ring of the gasket when compressed, but as long as you are small enough to do that it shouldn't matter.
I ended up with 2 more over time, but I could have likely got by with one more. I have 2 sitting in the hone rack, and usually use the last one as a pressure tester