correction, only the F5 emote deletes it, but you don't need to be in any sort of prone.

Tested it some more and it turns out that emoting just deletes your held item, me and my group have lost 2 guns so far from this.