Yep, you always need to use the retainer.
The step on the retainer loads the inner spring and pushes it down further, even on an installed height position.
As stated though, you have to measure the thickness of the top "flange" of the retainer and take that into account.