When running into Khelm, from the north side, I got major framerate drops/stutters, and it lasted through the whole time I was looting, fighting zombies, and generally just going through the town.
I think this may have been caused by the infected, loot, and then the deer in the forest right beside Khelm.
I'm playing on the release day Xbox One, and this lasted for a good part until I left Khelm (the worst in the north side of Khelm, when I just entered)