Page MenuHomeFeedback Tracker

Obscure compiler freeze error
Closed, ResolvedPublic

Description

DayZ Server Version 1.08.153276
So when loading certain mods with dayz server will freeze and never load when certain mod lists are added (see below where it stops in the console) seems to be some critical mass of something that these all or some of the mods contain, this issue has been plaguing the community since 1.08 with tons of different mods seeming to contribute to the issue.

12:46:42 !!! [Inputs] Cannot read inputs cfg "ExpansionProfile\Users\Survivor\Server.core.xml", parser error
12:46:42 [Inputs] Loading inputs cfg "bin/core.xml"
12:46:42 !!! [Inputs] Cannot load input preset "user", file does not exist
12:46:42 [Inputs] Exporting inputs cfg "ExpansionProfile\Users\Survivor\Server.core.xml"

So my testing process I found a breaking point with my snap heli mod where when it was added it would break
The first step was I commented out a few things didn't fix so I commented out everything to see if it was the cpp and it still froze tried a few things with the cpp then I tried removing files so it was just the cpp and it worked so I added back files till it broke and it was the UserActionComponent file then just for shits and giggles I added that file to the right folder (still commented out) tested again worked, then tried uncommenting everything it froze again so going based on the previous findings took a stab and tired fixing the extended part of the classes in snapheli and it still didn't work so I fixed a few classes in heroes and Bandits and bingo server would load . . .

Details

Severity
Crash
Resolution
Open
Reproducibility
Always
Operating System
Windows 10 x64
Operating System Version
Windows Server 2016
Category
Modding
Steps To Reproduce

Start command

start "Dayz Server" DayZServer_x64.exe -cpuCount=12 -exthreads=7 -instanceId=1 -config=serverDZExpansion.cfg -profiles=ExpansionProfile -port=2302 "-mod=@CF;@COT;@ExpansionLicensedExp;@ExpansionExp;@Summer_Chernarus;@WindstridesClothingPack;@Cl0udsMilGear;@MasssManyItemOverhaul;@MunghardsItempack;@ArmaWeaponPack;@ArmA2ACR;@DocsWeaponPack;@CPBWeapons;@AdvancedScopes;@BulletStacks;@CannabisPlus;@DrugsPLUS;@MuchStuff;@dboslots;@NehrPickup;@BMW525iE34;@BMWExpansion;@LandRoverDefender;@WorldWarGamingVehicles;@MuchCarKey;@ReduceVehicleDamageExp;@ExpansionCLExpanded;@HABEXP;@YNREXP;@AirRaid;@Banking;@Trader;@zSpawnSelection;@SnapHeliToHeliPad" -servermod=@SurvivorMissions -scrAllowFileWrite -noFilePatching -dologs -adminlog -freezecheck

Steam ID to mod name list
1559212036,@CF
1564026768,@COT
2178159769,@ExpansionLicensedExp
2178159764,@ExpansionExp
1644467354,@Summer_Chernarus
1797720064,@WindstridesClothingPack
1630943713,@Cl0udsMilGear
1566911166,@MasssManyItemOverhaul
1734713776,@MunghardsItempack
1793351435,@ArmaWeaponPack
2141780688,@ArmA2ACR
1864482799,@DocsWeaponPack
1705066179,@CPBWeapons
1932611410,@CannabisPlus
2170927235,@DrugsPLUS
1991570984,@MuchStuff
1869021368,@BMW525iE34
2004807460,@BMWExpansion
2107418217,@dboslots
2141350489,@NehrPickup
1912237302,@LandRoverDefender
2128058307,@ReduceVehicleDamageExp
2151717908,@WorldWarGamingVehicles
2143128974,@AdvancedScopes
1711179264,@BulletStacks
1832448183,@FlipTransport
2120002733,@MuchCarKey
2104303753,@HABEXP
2173711903,@ExpansionCLExpanded
2159122303,@YNREXP
2065443797,@AirRaid
1836257061,@Banking
1590841260,@Trader
2017605880,@zSpawnSelection
1988925918,@SurvivorMissions

@SnapHeliToHeliPad see attached PBOs rename and use the one labelled Not working and it will freeze, then rename and use the one labelled working and it won't freeze only difference in the files is the position of one file with commented out code "ActionConstructor.c"

Additional Information

Other Mod list that are known to cause the issue collected from reports in the community

@CF;@COT;@ExpansionLicensed;@Expansion;@RadiationZone;@RZEX;@ WindstridesClothingPack;@Cl0udsMilGear;@ MasssManyItemOverhaul;@ MunghardsItempack;@ HeroesAndBanditsEXP;@ArmaWeaponPack;@ArmA2ACR;@DocsWeaponPack;@ CPBWeapons;@MuchStuff;@CannabisPlus;@DrugsPLUS;@BMW525iE34;@LandRoverDefender;@ReduceVehicleDamage;@MuchCarKey_Expansion;@ExpansionCodeLocksExpanded;@Banking;@ Trader;@zSpawnSelection

and

1559212036    CF
1564026768    Community-Online-Tools
2116157322    DayZ-Expansion-Licensed
2116151222    DayZ-Expansion
1708571078    VPPAdminTools
1623711988    VanillaPlusPlusMap
2075831381    SchanaModParty
1680019590    Server_Information_Panel
1646187754    Code Lock
1878060278    PVEZ
2109061314    PVEZMarkersOnVPPMap
1710977250    BaseBuildingPlus
1991570984    MuchStuffPack
2132877776    MG_Base_Storage
1617874376    OP_BaseItems
2007175894    MoreDoors
1566911166    Mass'sManyItemOverhaul
1605653648    FIDOv PACK 4
1705066179    CPBWeapons
1797720064    WindstridesClothingPack
1748044748    BetterSuppressors
1590841260    Trader
1836257061    Banking
2067202485    SimpleBulletStacks
1932611410    CannabisPlus
2095424002    M1161 Growler ITV
1879540249    RaG_Hummer_Refresh
1756237462    [CrSk] GAZ-3309

and

@DayZ-Expansion;
@DayZ-Expansion-Licensed;
@CF;
@VPPAdminTools;
@VPPNotifications;
@AdvancedWorkbench;
@Atv_mung;@Banking;
@BuilderItems;
@CannabisPlus;
@CarsPlus;
@CJ187-Cars;
@Community-Online-Tools;
@CPBWeapons;
@Dayz_Navigation;
@dbo_slots;
@FIDOv PACK 4;
@JunkYardDog;
@MedicalAttention;
@MoreFood;
@MuchStuffPack;
@MunghardsItempack;
@Nehr_Pickup_Lada;
@Newbankmenu;
@PerishableFood;
@Radio;
@SQUAD MSF-C;
@ToxicZone;
@Trader;
@UltimateWeaponPack;
@WindstridesClothingPack;
@Zeroy-FishingZ;
@Zeroy-JeepWillys;
@Survivors_Hard;
@dbo_lada_lm;
@Gerphelius Zil-130;
@ChernarusRP AE86;
@AdvancedFurnitureMod;
@ExtraFarmingItems

and

@CF 
@Airdrop-Upgraded
 @VPPAdminTools 
@Community-Online-Tools 
@Specialist Weapon Pack 
@Docs_Weapon_Pack_Standalone 
@DayZ-Expansion-Licensed 
@DayZ-Expansion 
@MuchStuffPack 
@Trader 
@GoreZ 
@CheckIdentity 
@[Remastered] Arma Weapon Pack 
@BaseBuildingPlus 
@UsefulSuppressors 
@WindstridesClothingPack 
@Banking 
@Code Lock 
@Server_Information_Panel 
@BuilderItems 
@CPBWeapons 
@ASC ModPack 
@TraderFix_v2 
@CannabisPlus 
@Pen Pen Pirulin Weapon Pack 
@MVS_Fix 
@Sin City Test 
@Deceit 
@HellRetex Additions Weapons 
@HellRetex Mod 
@AwesomeArmbands

https://yourenotready.ca/wp-content/uploads/2020/08/Capture.png from futuresixx

I was also able to fix the issue by ensuring that as many modded classes had the appropriate extends in place and moving the files around into mods cause if you uncomment the mods it still freezes not till I move files around in both my Heroes and Bandits mod and the snapheli mod does it fix when snapheli is uncommented with the files in the matching folders.

Also, you can see some discussion in the DayZ Modders discord in the channels #Radiation-Zone and #community-framework

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

Just because I want to maybe provide as much information as possible I uploaded the dump files for the DayZ Server Files three files exist in the link as well as attached the EXE to vs code debugging to try and get anything at all.
StartUpCommand_BadFreeze.txt The startup script with the mods that get the server to freeze
DayZServer_x64BadFrozen.DMP
DayZServer_x64_2020-08-25_17-12-53_BadFreeze.RPT
callstack_BadFreeze.PNG
VSCodeDebugLogs_BadFreeze.txt

For a benchmark same mod list minus 2 mods that I removed to allow the server to overcome the freeze.
StartUpCommand_GoodPastFreeze.txt
VSCodeDebugLogs_GoodPastFreeze.txt
callstack_GoodPastFreeze.PNG
DayZServer_x64_GoodAtServerCore.DMP
DayZServer_x64_GoodAfterServerCore.DMP
DayZServer_x64_2020-08-25_17-15-59_GoodPastFreeze.RPT

Mods to ID List.txt this is just a list of mod names to their workshop IDs to help Identify the mods more clearly

https://drive.google.com/drive/folders/1pFdWLGphzmPtvi57AZ8o0vmLGv-4-5O2?usp=sharing

This issue is worse with the 1.09 patch. I needed to remove 5 more mods (32 down to 27) to get my server to boot.

Also, update for anyone following this ticket and for more information for BI

I found a way to mitigate server core freeze a bit more, moving code to 3_Game, well at least any code that can be moved, I specifically moved all my Config loader classes to 3_Game for HeroesAndBandits and was able to get a handful of mods re-added,

this was thought when I was remembering errors in the logs about bad type JSONFileLoader error with the freeze and knowing that JSONFileLoader class is in 3_game, and realizing that errors in 3_game would throw but none in 4_world when the freeze is happening and coming to the conclusion that the compiler was compiling each module separately so possibly moving code to 3_game would mean that it wouldn't reach its limit as quickly

So bottom line the more code that modders can put in other scopes EG 5_Mission, and 3_Game, the more code we will be able to have in 4_World so move your code if you can!

AWG added a subscriber: AWG.Sep 9 2020, 7:44 PM

I had PvDCustombisbleZombies Follow your advice Daemonforge and He was able to push his mod to the limts with the code base. and...... :D What was not working is now :D BI Please look into this issue as a HotFix or for 1.10 PLLLLLLLLEEEEEEEEEAAAAAAAASSSSSEEEEE :D

Lewk added a subscriber: Lewk.Oct 21 2020, 3:29 PM
Lewk added a comment.EditedOct 21 2020, 3:34 PM

Always get this error when combining a lot of mods that have many scripts in 4_World, such as:

@CF 
@Community-Online-Tools
@DayZ-Expansion-Licensed
@DayZ-Expansion
@Cl0ud's Military Gear
@Modular Vest System
@Base Fortifications
@Base Furniture Mods 3.0
@Vote-Skip-Night
@VPPAdminTools
@[Remastered] Arma Weapon Pack
@CPBWeapons
@Trader 
@Trader logging
@CFDZPlugin
@CFDZAddIn-Expansion
@GrosTon1s Servers Tracker 
@Server Watcher

These are some mods that appear to have many World scripts, typically combing these reproduces the issue

@CF 
@Community-Online-Tools
@DayZ-Expansion-Licensed
@DayZ-Expansion
@CFDZPlugin
@CFDZAddIn-Expansion
@GrosTon1s Servers Tracker 
@Server Watcher
AWG added a comment.Nov 12 2020, 8:22 PM

Has this been fixed in 1.10 Experimental yet? I know the default variable bug was fixed, but wasn't sure if this was. This issue seems to pop up randomly, I've been running the following mods for a while, and during server restarts, this error will sometimes popup and cause the server to freeze, but if I restart it 5-6 times then it will start normally. This happens even though there were no updates to any of the mods or the game itself, so I don't get it.

@CF
@Community-Online-Tools
@DayZ-Expansion
@DayZ-Expansion-Licensed
@VPPNotifications
@zSpawnSelection
@KillReward
@BaseBuildingPlus
@Expansion Code Locks Expanded
@Modular Vest System
@SQUAD MSF-C
@Munghardsitempack
@WeaponReduxPack
@Remastered Arma Weapon Pack
@Cl0uds Military Gear
@Cl0uds Military Gear Expansion
@FIDOv PACK 4
@MoreGuns
@MassManyItemOverhaul
@CPBWeapons
@Advanced Weapon Scopes
@RaG_Hummer_Refresh
@IRP-Land-Rover-Defender-110
@Dingo_Truck
@MRAP 4x4
@Devils Dandruff
@MG_Colorful_Lights
@No-Vehicle-Players-Damage
@BetterSuppressors
@BulletStacksPlusPlus
@FlipTransport
@Banking
@HeliPad
@AWG Server Mod
@dbo_surfaces
@InventoryPlus
@InventoryPlusPlus
@Trader
@TraderFixesAndFeatures
@Time Is Money
This comment was removed by acdemiralp.

Ok 1.10 has made the so so much worse. I have removed 5 mods already (removed 7 with 1.09) and I still can't boot

This comment was removed by QripTiiieee.
winter removed a subscriber: winter.Nov 19 2020, 5:12 PM
Liven added a subscriber: Liven.Nov 19 2020, 10:08 PM

The problem still happen with 1.10.
Any news?

This comment was removed by acdemiralp.
Geez changed the task status from New to Assigned.Nov 20 2020, 9:42 AM
Geez added a subscriber: Geez.Nov 20 2020, 3:03 PM

Hello everyone.
We have been successful in producing this issue by combining 48 mods on the 1.10 version and we are currently looking into what can be done.
Regards,
Geez

THANKS!!!
Also Geez just so you know cause I have done a lot to work on this an some of all the information is scattered a quick overview of my findings

This issue seems to be module specific I haven't been able to replicate the issue in other modules yet but for sure in 4_World this issue is reproducible, it seems that not all scripts are created equal in this either, I find scripts that use the JSONFileLoader Class is one of the more impactful in this issue. Also, I have noticed that most types of compiler errors won't throw in 4_world if the freeze is happening as well, which is why I belive its an issue with a pre compiler?? (if there is one, don't know enough about Enfusion) but that is just a best guess

lava76 added a comment.EditedNov 23 2020, 1:49 PM

Moving code from 4_World into 3_Game definitely helps mitigate the problem, as does getting rid of any JsonFileLoader usage. What I also found helps is grouping all mods that use JsonFileLoader and load them last, after other mods. Hopefully the cause(s) for these freezes can be found and a proper fix be implemented.

daemonforge added a comment.EditedNov 25 2020, 6:31 PM

Geez You don't need 48 mods to replicate anymore you should be able to replicate it with just this one mod attached.
This mod contains just over 1400 Classes in 4_world all using the JSONFileLoader I am going to continue to test and narrow down or find more things that cause the freeze. But moving all these classes to 3_Game doesn't cause the freeze.

You can also use this one has more classes (1900) but there it's not just a single class, I am using [NonSerialized()] and Params5 as well as JSONFileLoader.

Xairo added a subscriber: Xairo.Nov 26 2020, 3:49 AM
Geez changed the task status from Assigned to Need More Info.Dec 4 2020, 2:03 PM

Hello everyone.
The latest experimental that will be released today will have a potential fix for the issue. Can you please let us know if everything works correctly?
Regards,
Geez

Gramps added a subscriber: Gramps.Dec 4 2020, 4:23 PM

No luck here! I can't even add one more mod to my server running this latest build of exp.

  • Gramps

@Geez So it seems that some stuff is fixed all my test mods that could cause the freeze on their own don't anymore so that's good, and combining those with larger mods(*cough* *cough* expansion) they work fine but I am still getting some mod combinations that are causing the freeze I am working on gathering the smallest number of mod combinations so I can report them here :)

One of the smallest combinations so far

"-mod=@CF-Test;@COT-Test;@DayZ-Expansion;@DayZ-Expansion-Core;@DayZ-Expansion-Licensed;@DayZ-Expansion-Vehicles;@Trader;@Namalsk Survival;@MasssManyItemOverhaul;@CannabisPlus;@MuchStuffPack;@HaB Experimental"
lava76 added a comment.Dec 4 2020, 6:52 PM

No change with latest Experimental for me unfortunately.

The feedback here boys is impeccable! We can fix this! Shame its not in this push to experimental though.

Im still getting stuck on core.xml
I have only 10 mods on test server.

@CF
@Community-Online-Tools
@VPPAdminTools
@DayZ-Expansion
@DayZ-Expansion-Core-
@DayZ-Expansion-Vehicles-
@DayZ-Expansion-Licensed
@MasssManyItemOverhaul
@WindstridesClothingPack
@Cl0uds Military Gear
Jest added a subscriber: Jest.Dec 5 2020, 5:47 PM
Geez added a comment.Dec 7 2020, 10:31 AM

Thank you for your feedback.
We are going to continue working on this and possibly push more fixes in the experimental updates. I will keep you posted on this.
Regards,
Geez

Geez added a comment.Dec 7 2020, 1:44 PM

Hello again.
New experimental is out, can you please give us some feedback again?
Regards,
Geez

Used the following mods directly from the '!Workshop' directory.

@CF
@Community-Online-Tools
@DayZ-Expansion-Core-
@DayZ-Expansion-Licensed-
@DayZ-Expansion-Vehicles-
@DayZ-Expansion-
@Trader
@Banking
@VPPAdminTools
@MuchCarKey
@MuchStuffPack
@ToxicZone
@Namalsk Island
@Namalsk Survival
@Namalsk_Igloo
@GRW ER7 Gauss Rifle
@Heroes And Bandits
@VPPNotifications
@zSpawnSelection
@Vehicle3PP
@SchanaModParty
@Notes
@NoVehicleDamage
@NoExplodingHelis
@No Shoe Damage
@BreachingCharge
@HellRetex Mod

Tested this configuration and can confirm it does not work in stable but it does load in experimental.

Added @BaseBuildingPlus with a second test, in stable, it freezes on "server core", in experimental it outputs script errors (possibly due to conflicting mods).

Just tested with @DafixCz's mod list. Again, in Stable, hard freezes on "server core", in the new Experimental patch, it goes past that stage.

Verified mods are loaded.

Gramps added a comment.EditedDec 7 2020, 4:00 PM

Great news! I was able to add multiple mods back to my list without getting experimental to freeze!
I'm only able to run 33 mods on stable, Im now up to 84 mods on experimental and it hasn't froze yet! :)

So far can't cause the freeze but a new error has occurred once I got to this point of mods specifically after adding VPPAdminTools

"-mod=@CF;@VPPAdminTools;@UniversalApi;@Community-Online-Tools;@DayZ-Expansion;@DayZ-Expansion-Core;@DayZ-Expansion-Licensed;@DayZ-Expansion-Vehicles;@Trader;@Namalsk Survival;@MasssManyItemOverhaul;@CannabisPlus;@MuchStuffPack;@P2PTrader;@ECLE Expermental;@CarCover;@Metallurgy&Forging;@HaB Experimental;@Basic Map;@MuchCarKey;@GasMaskAddons;@FarmingPlus+;@BaseBuildingPlus;@Code Lock;@RadZone;"

the error is

SCRIPT    (E): NULL pointer to instance
Function: '#InitGlobals'
Stack trace:
Food\scripts\4_World/useractions\drugself.c:43
SCRIPT    (E): NULL pointer to instance
Function: '#InitGlobals'
Stack trace:
Food\scripts\4_World/useractions\drugself.c:43
Gramps added a comment.EditedDec 7 2020, 6:41 PM

I did get that #InitGlobals error around mod 73.. However my stack trace was through a different mod (Coca's No Vehicle Damage Expansion).
I removed the most recently added mod (Metallurgy&Forging), and was able to add an additional 12 mods, bringing my total up to 85 (from 33 in stable)!
I did have a couple mods that would cause the server to crash at server.core.xml, instead of freeze.
So I removed those mods and kept adding others that would not cause a crash.

Im thinking either: the server.core still happens, and if so, now crashes instead of freezes; OR those tested mods are just incompatible with 1.10 or experimental, crashing the server.
Either way, Im able to load MANY more mods with this build! :)

Latest Experimental looks like it did the trick. That's a huge relief. It allowed me to add the three mods back in that I previously had to deactivate, plus some more.

@Geez if in the end it's something we(vpp) have to change, im fine making the change, but I must understand what causes this, so I can avoid and fix it.

@ukulelekid713 it's not related to VPP or any other single mod.

@Gramps Can you provide a mod list for the #InitGlobals error?

@lava76 Can you trigger any issue with your mod list in EXP?

No issue for me with the mods I run in Experimental. I would be very happy (well not only me I'm sure) if the current fix in Experimental would appear in stable soon.

Agreed. Even if this fix can be improved in the future, I'd rather get the version on experimental out to stable before Xmas

Ukulelekid713, I no longer have the list of mods I tested above, however I DO remember that a specific mod (@Metallurgy&Forging) created the #InitGlobals error mentioned above. I removed the offending mod and was able to add a few more, until I just gave up around 85 mods as the startup process was not freezing on server core. I assumed it was something else introduced in this exp build that caused this new #InitGlobals error making specific mods incompatible.
I agree though, better sooner than later! As of now, I'm still sweating on every mod update, will my server freeze or nah?

This comment was removed by acdemiralp.

Could we please get some information what did you guys fix internally? (it does not need to be technical line by line, just some heads up on something we can relate within EnforceScript) Might clue us in on looking for an easy way to reproduce. Because right now i am stuck with adding a single "if statement" or declaring something "new" of a custom typename causing crash during 4_World module compile
I have a 100% reproduceable method right now, but it must include a large list of mods; and the issue is presented based on combination of scripts that have no correlation from one mod to another (on surface level aka Enforce Script modules)

-mod=@CF;@VPPNotifications;@VPPAdminTools;@UniversalApi;@Community-Online-Tools;@DayZ-Expansion-;@DayZ-Expansion-Core-;@DayZ-Expansion-Licensed-;@DayZ-Expansion-Vehicles-;@Trader;@Namalsk Survival;@MasssManyItemOverhaul;@CannabisPlus;@MuchStuffPack;@P2PTrader;@ECLE Expermental;@CarCover;@Metallurgy&Forging;@HaB Experimental;@Basic Map;@MuchCarKey;@GasMaskAddons;@FarmingPlus+;@BaseBuildingPlus;@Code Lock;@RadZone;@Server_Information_Panel;@SQUAD MSF-C;@KillFeed

Add the following mod to that list above @KillFeed will cause crash (sometimes along the crash the Function: '#InitGlobals' stack trace is thrown ) when 4_World is about to load. I have pinned it down to the exact script causing the crashes within the mod @VPPAdminTools, however it produces inconsistent outcomes, meaning if you remove @KillFeed the @VPPAdminTools codebase will have no issues whatsoever and vise versa.

Going deeper within the script class in question:
The specific code in question is FileSerializer although it is utilized in many modules and in other instances within the 4_World module, it still causes issues. I cannot describe why because the issue seems to be arbitrary, mostly based on mod load order and type mod hence why its so darn hard to create a combination of short mods to trigger the behavior.

One consistent result thru the testing so far: all the issues stem from 4_World module all mods in question that cause the crash / #InitGlobals stack trace have edited that module worldScriptModule and when unlinking a specific mod out from loading its modification to that module it stops the crashing and resolves compiler issues no matter which mod you remove, at least either (in this context) @Killfeed or @VPPAdminTools

The strange thing is, the correlation between the two mods in question have no relation as in: @Killfeed makes modifications to vanilla classes such as recipes and player actions, the other mod @VPPAdminTools creates its own custom types and uses FileSerializer within multiple class & instances. Remember this is all within within 4_world module. The addition of more mods which override and add-onto this module in question, causes further more chance of this issue occurring.

Within the codebase of the mod @VPPAdminTools i was able to reach a point where commenting a few specific lines would fix the issue, the code in question is ALWAYS 100% related to ANY type of Serialization
i.e; declaring and using ParamX Param1-7 & class FileSerializer remeber, not only declaring the class but actually using it e.g;

Param1<string> data;
if (!ctx.Read(data))  <<<--- READ = EXPLODE
{
  //do something
}

consistent so far right? Yea because all these types link back to Serializer engine class with Read Write proto functions

Using the class in question as shown below will not cause any issues

GetRPCManager().SendRPC("RPC_MenuObjectManager","HandleSetsData",  new Param1<ref BuildingSet>(buildSet), true, sender);

Notice why it works just fine, results in no crashes because Read(); or Write(); was not used.
new Param1<ref BuildingSet>(buildSet)

I have attached a sample snippet that is documented on where and how it causes a crash (must use the same mod list i mention above. order too)

Geez added a comment.Dec 16 2020, 1:48 PM

Could we please get some information what did you guys fix internally? (it does not need to be technical line by line, just some heads up on something we can relate within EnforceScript) Might clue us in on looking for an easy way to reproduce. Because right now i am stuck with adding a single "if statement" or declaring something "new" of a custom typename causing crash during 4_World module compile
I have a 100% reproduceable method right now, but it must include a large list of mods; and the issue is presented based on combination of scripts that have no correlation from one mod to another (on surface level aka Enforce Script modules)

-mod=@CF;@VPPNotifications;@VPPAdminTools;@UniversalApi;@Community-Online-Tools;@DayZ-Expansion-;@DayZ-Expansion-Core-;@DayZ-Expansion-Licensed-;@DayZ-Expansion-Vehicles-;@Trader;@Namalsk Survival;@MasssManyItemOverhaul;@CannabisPlus;@MuchStuffPack;@P2PTrader;@ECLE Expermental;@CarCover;@Metallurgy&Forging;@HaB Experimental;@Basic Map;@MuchCarKey;@GasMaskAddons;@FarmingPlus+;@BaseBuildingPlus;@Code Lock;@RadZone;@Server_Information_Panel;@SQUAD MSF-C;@KillFeed

Add the following mod to that list above @KillFeed will cause crash (sometimes along the crash the Function: '#InitGlobals' stack trace is thrown ) when 4_World is about to load. I have pinned it down to the exact script causing the crashes within the mod @VPPAdminTools, however it produces inconsistent outcomes, meaning if you remove @KillFeed the @VPPAdminTools codebase will have no issues whatsoever and vise versa.

Going deeper within the script class in question:
The specific code in question is FileSerializer although it is utilized in many modules and in other instances within the 4_World module, it still causes issues. I cannot describe why because the issue seems to be arbitrary, mostly based on mod load order and type mod hence why its so darn hard to create a combination of short mods to trigger the behavior.

One consistent result thru the testing so far: all the issues stem from 4_World module all mods in question that cause the crash / #InitGlobals stack trace have edited that module worldScriptModule and when unlinking a specific mod out from loading its modification to that module it stops the crashing and resolves compiler issues no matter which mod you remove, at least either (in this context) @Killfeed or @VPPAdminTools

The strange thing is, the correlation between the two mods in question have no relation as in: @Killfeed makes modifications to vanilla classes such as recipes and player actions, the other mod @VPPAdminTools creates its own custom types and uses FileSerializer within multiple class & instances. Remember this is all within within 4_world module. The addition of more mods which override and add-onto this module in question, causes further more chance of this issue occurring.

Within the codebase of the mod @VPPAdminTools i was able to reach a point where commenting a few specific lines would fix the issue, the code in question is ALWAYS 100% related to ANY type of Serialization
i.e; declaring and using ParamX Param1-7 & class FileSerializer remeber, not only declaring the class but actually using it e.g;

Param1<string> data;
if (!ctx.Read(data))  <<<--- READ = EXPLODE
{
  //do something
}

consistent so far right? Yea because all these types link back to Serializer engine class with Read Write proto functions

Using the class in question as shown below will not cause any issues

GetRPCManager().SendRPC("RPC_MenuObjectManager","HandleSetsData",  new Param1<ref BuildingSet>(buildSet), true, sender);

Notice why it works just fine, results in no crashes because Read(); or Write(); was not used.
new Param1<ref BuildingSet>(buildSet)

I have attached a sample snippet that is documented on where and how it causes a crash (must use the same mod list i mention above. order too)

Hello markkoky44.

  1. Upgrade of "budget" for variables which were exceeded by all those 50 mods where together with several inappropriate declaration in several of mods, which in fact caused dead lock instead fair warning of budget exceed was added do core compatibility.
  1. Your question is quite complicated and you are also trying to progress ad-hoc, so it is not clear where to focus honestly.
  • seems you are breaking binary stream there, which will always cause crash
  • the worse in network environment (RPC usage)
  • when serializing you have to perfectly match both Read and Write, mean: always fully deserialize 100% same content which was serialized
  • you should start refactoring this system into clear parts for serialization and networking where you will be able to log what is happening and handling + spawning

We hope you don't take this personally - but to "fix this" we would need to get full scope of what you are trying to achieve and most probably design and write brand new solution.

Regards,
Geez

Hello Geez,

Thanks for the feedback. Refactor is the route indeed however sharing this I figured it could possibly help in resolving or at least finding better leads to *crash on compile* bug. It is unknown what other code in the future might face the same issues (with other user created content); rewriting it again is an option all the way until you run out of things to do because the bug happens. Although my implementation shown in the snippet is mediocre, it seems odd that the compiler fails to load it although its fairly basic in what it attempts to achieve (this implementation has been thru several patches since 1.03.xxx).

Thanks!

@Geez any ETA on when the current fix in Experimental will appear in stable?

Soon I hope! I had to remove yet another mod this week! :(
I was trying to wait, but now Im seriously thinking of switching to exp build!

Geez added a comment.Dec 18 2020, 1:14 PM

@Geez any ETA on when the current fix in Experimental will appear in stable?

Unfortunately, there is no info I can share at the moment.
Regards,
Geez

Seeing that blog post today, seems you guys are done for the year? So I guess I'll be teaching my players how to move to exp branch for xmas! XD

NATION added a subscriber: NATION.Dec 18 2020, 5:41 PM
This comment was removed by NATION.

I just came by to say that I had to drop yet ANOTHER mod today because of this issue... What's the holdup on pushing this MUCH needed fix?

webnebular removed a subscriber: webnebular.
Zezior added a subscriber: Zezior.Dec 31 2020, 1:56 PM

Currently using these mods in this order and I get 90% boot chance successful as sometimes core freezes then after 1-2 reboots its successful.
I also wanted to add cannabisplus, drugsplus, muchstuffpack, toxiczone but if I add any of them the server refuses to boot and is completely stuck.
Will this experimental push potentially fix this? and if possible could anyone replicate to see if it is related to this issue?
Also not seen anyone talk about DayZEditor Loader here so idk if that could be making an impact.

@Summer_Chernarus
@CF
@Community-Online-Tools
@DayZ-Expansion
@DayZ-Expansion-licenced
@DayZ-Expansion-core-
@DayZ-Expansion-vehicles-
@DayZEditor Loader
@BaseBuildingPlus
@Trader
@Banking
@Code Lock
@ModularVest System
@Breachingcharge
@VanillaPlusPlusMap
@Cheytac M200 from ArmA 2 Port
@MoreDoors
@TraderFixesAndFeatures
@MoreGuns
@Mortys Weapons
@zSpawnSelection
@zSpawnSelBaseAdd
@plus200_fix_110
@WeaponReduxPack
@[Remastered] Arma Weapon Pack
@dbo_surfaces
@CFDZPlugin

@Zezior yes the current experimental version fixes in the issue. You can change which fiesl your are using via steam and then run a test with the same mod setup. People above loaded up to 84 mods before they got bored, but there may still be some extreme cases where it fails. Will have to see properly when its on stable

I don't want to be a nag, but please can this get bundled into a hotfix and not held off until 1.11. Players don't understand this and are turning the blame towards owners. Unfounded sure, but a hotfix will get everyone happy again, especially with the current player numbers.

Lewk added a comment.Jan 12 2021, 1:42 PM

I also agree with this. Just because DayZ hadn't updated doesn't mean mods aren't updating.

With each Expansion update we have to remove more mods because of this error.

I am sorry to those who are dealing with this, at least there is a solution to the issue somewhere on the horizon. I am looking forward to not having people believe that it's directly related to our mod(s). It has been hammering me for months now.

This comment was removed by acdemiralp.
Geez closed this task as Resolved.Jan 20 2021, 11:19 AM
Geez claimed this task.

Hello everyone.
The fix will be pushed with the 1.11 stable version.
Regards,
Geez

This comment was removed by acdemiralp.
Gramps added a comment.EditedOct 4 2021, 5:40 PM

Getting the same here, InitGlobals server crash when using "too many mods" using scripts. Seems like it's become a well known issue as of 1.14 but I have not seen a new ticket on this matter. One of my servers is back to having to remove a mod or two with every new mod and/or game update. I was running a Namalsk server with up to 51 mods, but have since 1.14 had to scale them back to 45 mods total.

This comment was removed by acdemiralp.
Setman added a subscriber: Setman.Oct 13 2021, 12:36 AM

Having the same issue, can only load 33 mods. Been running Expansion for awhile now with no issues, but since last up date stopped working

@Geez

Sadly the issue cameback with a new error as you saw. Is there anything on your side you might be able to share with us so modders can try to find a work around ?

Setman added a subscriber: mass.Oct 14 2021, 12:29 AM

This is what I have in the start line
-instanceId=1 -config=serverDZ.cfg -port=2702 -profiles=profiles -adminlog -freezecheck -dologs "-mod=@CF;@CodeLock;@Community-Online-Tools;@Ear-Plugs;@CodeLock;@CannabisPlus;@DabsFramework;@BaseBuildingPlus;@FlipTransport;@DayZ-Expansion-Licensed;@DayZ-Expansion-Bundle;@KODWeapons;@Banking;@DrugsPLUS;@Trader;@TraderFixesAndFeatures;@VanillaPlusPlusMap;@BuildAnywhere_v3;@Garage;@InventoryPlusPlus;@RaG_Hummer_Armored;@SnapHeliToHeliPad;@TechsWeaponMod;@MunghardsItempack;@mass'sManyItemOverhaul;@RaG_MTVR;@[Remastered] Arma Weapon Pack;@Unlimited Stamina;@MRTsMREs;@TrailBruizerSUV;@ZomBerryAdminTools;" "-serverMod=@SurvivorMissions;@NoExplodingHelis;@FixVehiclePositions;"

I have tried it with and without the "" and also moving the dologs and profiles to the end of the line after the mods, still does the same
As soon as I try to add any of these mods @TeddysWeaponPack , @Zeroy-FishingZ , @CPBWeapons or most any other ones it crashes right away

Also tried older version's of Expansion.ChernarusPlus in case it was in there, but it still crashed. Of course did fresh install of game and mods with no help.

Update
It seems like when @DayZ-Expansion-Licensed;@DayZ-Expansion-Bundle; just got updated I am now able to add more mods for some reason. Works great now.

Update
It seems like when @DayZ-Expansion-Licensed;@DayZ-Expansion-Bundle; just got updated I am now able to add more mods for some reason. Works great now.

Roughly 5-10 script classes were removed in the last Expansion update but they will be coming back in another update.

I noticed since a few days ago with the update I am no longer able to run my server. If I try to add anymore than these, it won't start.
"-mod=@CF;@Community-Online-Tools;@DayZ-Expansion-Licensed;@DayZ-Expansion-Bundle;@DabsFramework;@CannabisPlus;@DrugsPLUS;@CodeLock;@BuildAnywhere_v3;@FlipTransport;@InventoryPlusPlus;@mass'sManyItemOverhaul;@MRTsMREs;@MunghardsItempack;@SnapHeliToHeliPad;@RaG_Hummer_Armored;@Banking;@BaseBuildingPlus;@Zeroy-FishingZ;@TechsWeaponMod;@Trader;@TraderFixesAndFeatures;@TrailBruizerSUV;@Breachingcharge;@VanillaPlusPlusMap;@ZomBerryAdminTools;@UnlimitedStamina;" "-serverMod=@Survivor Missions;@COCAs_NoVehicleDamageMOD_Expansion;@FixVehiclePositions;@NoExplodingHelis;"

Yes this needs to be reopened as the issue is back with a vengeance mostly with using Expansion I get the server to boot with these mods,..

@DeerIsle;@VPPAdminTools;@CF;@Dabs Framework;@Community-Online-Tools;@DayZ-Expansion-Core;@DayZ-Expansion-Licensed;@DayZ-Expansion;@DayZ-Expansion-Vehicles;@DayZ-Expansion-Book;@DayZ-Expansion-Market;@DayZ-Expansion-Name-Tags;@PersistentDoors;@Base Fortifications;@Code Lock;@BaseBuildingPlus;@BulletStacksPlusPlus;@[Remastered] Arma Weapon Pack;@Modular Vest System;@VIRUS;@GOLD Nuggets;@Rubber Bullets;@CannabisPlus;@DrugsPLUS;@Mercedes W220 S65;@FlipTransport;@dbo_surfaces_DI;

But if I was to add any others for example @Air Strike, @MunghardsItemPack, @MuchstuffPack, @OP_BaseItems,.. etc etc.. server will not boot,

Having the ability to add larger script baring mods together with expansion is needed to at least accommodate 60+ mods including Expansions mods and this script limit issue shouldnt be an issue anymore, The issue has been around since 1.13 at the least, A much higher script limit is drastically needed asap, I will also post this information in other relevant tasks to give wider acknowledgement in hopes a fix will arrive before the year is finished.