After more playing around with this I have discovered the problem. In optics mode mouse sensitivity is reduced making it slower to pull the barrel back down.
Not sure how this would be easily fixed. I think I will just +1 a "too many recoils" post.