After trying Accolyte's suggestion, my wife and I did some intentional server hoping. We were able to connect 100% of the servers we both tried. I consider this fixed for me now.

Oh, we did the port fix in the launcher, not the router change.

My wife and I were having the same issue, as recently as version 47. The port adjustment for the launcher seems to have greatly improved our chances. Not perfect yet, but now it only happens once in 10 tries instead of 9.9 in 10.