When climbing down a ladder the hands aren't grabbing the handles.
If you now stop on the ladder the hands will automaticly float down to the handles, from there on the positioning works.

Didn't try for climbing up a ladder.

Played yesterday on stress test 2-1 server