You can scroll actions and right-click to remove the symbol, but if you're doing a shoot out in PvP (or also against the AI), every second counts.
I also don't think this should be argued for as "pain effect, hindering you to react properly" - this should be done in terms of PP effects (as it is already done to some extent). :-)