This happened to me today, too. But you don't have to do all of these steps mentioned above:
Just right-click on a flare and select "Eject" (yes, only on the bare flare). Now you have a stackable flare because now there's a number ("1") on the icon. The maximum amount of flares in one stack is 5.