"They damage you even when they aren't attacking.
If that is on purpose, then you are failing at making this realistic in any sense whatsoever.
Make them a threat, yes. But don't make them to the point where it's not even worth engaging at all. "
This.
Zombies running around an entire house in a half a second without making a single footstep sound, then attacking you with kung-fu chain-punch assaults while falling backwards from your axe strike. Taking three magnum shots to the chest without ever bleeding out (guess they're undead after all). But being too dumb to smash in a window or door. No blocking, no horizontal strikes, no fluid combat motions, no dodging, nothing. Just the same old same old, but "harder".
A single infected shouldn't pose a challenge to a fully geared survivor with an axe at all. A group (three or four infected) should pose a serious danger. But if you wanted all of these actual gameplay mechanics, that would actually require some effort to develop, like a melee system....