Seems that disabling the Resizable Bar support and Above 4G Decoding in my bios resolves this issue entirely!

I did some further testing on this by putting in my old 4070Ti and booted up the game, everything ran perfectly fine, so on that basis I would say it's isolated to the 5080 GPU, however, I initially ran the 5080 with my i9 for some weeks before I upgraded the motherboard / CPU / RAM too, so it seems it's a combination of the 9800x3d along with the 5080 which causes the issue.