Certain ladders in harbors are unusable presumably because they clip through terrain.
Pressing F on the prompt does nothing.
Here is a video of one of these ladders: https://youtu.be/1rwZxX_WVIM
I did a cursory look around Everon and there are 3 more like this, marked on maps in the album below with the small red dot. I did not check Arland.