Whenever the inventory is open, the performance drops a bit, in terms of FPS, of around 5-10 frames, at least for me. I've noticed that when I scroll, however, the GPU usage drops, as seen in the video, and the FPS drops to <20 also. I'm not sure if it's a problem with my settings, but I've noticed this FPS drop even when servers load in the server browser.

https://www.youtube.com/watch?v=0-p-CaJo-_U Here's the video.