After player count hits around 25, wait for it to hit top of an in game hour and server desync jumps to 100000 for all players for roughly 10 seconds.

I would say it is due to how global sounds are produced in game. Before the 1.26 patch(when the global church and door sounds were added). When a global sound is called, it checks the position of every player to relay back to say if needs to play these sounds or not. As we can see this is a terrible thing for server performance as every in game hour, it has to see where every player is in relation to every single church on the Altis Map(what is the count? around 160 or so) and and that would be 160 or so scripts per player running at once to check player distance to the churches, same for doors opening and closing. Just my two cents, but what would I know.

We have an enterprise level server running dual E5's, 1Gbps redundant up and download speeds, along with 128 Gb of RAM and optimized bandwidth settings.