I have the same issue. I'll toggle zoom, put up iron sights, zoom will go away, and then I have to press the zoom toggle TWICE (probably due to a variable in the code that doesn't get reset when using iron sights).
Also, toggle zoom doesn't work well when using vehicles.