I know how this is caused. This happens when you join a friend through steam friends. When joining via the friends tab on the server browser, this doesn't happen