99% of the work is done the second that valve cracks off the seat. The rest is just time to let the "event" take place. We like to think we know what's happening in the ex port, but we really don't. Flow bench is all but meaningless when it comes to ex ports other than for cam design and selling heads.
I was sorta amazed at the slow motion video on YouTube of the Briggs & Stratton engine with the plexiglass head. Of course it has practically zero in common with any car engine, but you can still see how almost instantaneously the exhaust exits the chamber. And like you said, the rest of the event is just time to get that last little bit out to not contaminate the incoming mixture. Still, getting that last little bit out makes a big difference in the next combustion cycle. And despite some of the tests showing how little of a difference headers make in power production, there's plenty of data showing the effects of a well designed system in a drag application. Not so much on most street applications though.