Character "vectorup" while climbing a ladder is always equal to [0,0,1] regardless of actual orientation of ladder, causing visual animation glitch where the character doesn't grip the ladder but instead "climbs on air".
While the ladder memory points are used for the path of the climbing, they aren't used to decided the "vectorup" of the character climbing.

There's a possible script solution for this:
_building = nearestBuilding player;
_ladderStart = _building modelToWorld (_building selectionPosition "Ladder_1_S");
_ladderEnd = _building modelToWorld (_building selectionPosition "Ladder_1_E");
_vectorUp = _ladderEnd vectorDiff _ladderStart;
player setVectorUp _vectorUp;

But the only event handler suitable for such case is the AnimChanged, and there's still the issue of a hackish method to get the ladder the character is climbing based on which one is the closest.