Since my buddies don't have this problem, i suspect my network provider is at fault for giving me erratic bandwidth and packet loss (i noticed an overall reduced network performance in the evening hours).
But that still doesn't explain the absolute broken downstream (upstream seems to work, as all of my actions are send to the server just fine). Other applications, just as Teamspeak or Skype works just fine.
And if, for whatever reason, my connection to a server is broken, the game should not crash but giv me a "session lost" kick or something.
I tried different servers too, with the same results.