Fact is I've serial killed about a dozen of people in Green Mountain lately.
They come up the ladder warily, reach the top and see me through third person, then hastily try to escape.... which is futile
Generally, when on a ladder, you're pretty much screwed.
My suggestion: make it possible to climb a ladder with pistol in hands.
When clicking right-click (scope / ironsights), the character stops climbing and turns his body while holding on to the ladder with one hand.
It could also be enabled to climb up, though slowly, and for every spoke do like a massive sway of head and pistol hand.
Besides, aim should be reduced (due to only holding in one hand)
But this way, a player is not a sitting duck when climbing up a ladder.
It is also a great incentive to carry a pistol in your inventory.