Could be related to router (FRITZ!Box 7430) or firewall/port forwarding.
Or some faulty auto limitation of connections to avoid overload/auto banning issue.
Tried different settings in Steam but seem unrelated.
Never had issues with the ingame server browser listing servers very slowly.