Or you convert to a hydraulic booster
They use a pressurized nitrogen cannister to store enough braking energy for a couple stops, if you lose your power steering belt, or something. Then, once you bring the system back up to pressure, it re-charges the reserve cannister
If you want to feel the reserve on your vacuum brakes, shut the car off after running it, then press the pedal. It moves easy, like usual. Press it a couple more times, and you'll see what happens as the vacuum reserve is used up, and you have not assist. It's not a bad thing to know what it feels like, just in case your engine ever stalls while you're driving it, and you have to stop without the assist.
The loss of engine vacuum in certain situations is certainly one issue. As is the fact that some very aggressive cams produce very little vacuum, to begin with. But the biggest "day to day" thing that I don't like about vacuum brakes is their inconsistency. Since they depend on engine vacuum, which varies constantly, the braking is never consistent
Joe