Noticed this when weapon is up (spacebar). However, if you bring down your weapon, switching can be done without it stopping your character.