Page MenuHomeFeedback Tracker
Create Task
Maniphest T180210

Dayz Server Linux Stable 1.24 JSON files write corruption issues
Feedback, UrgentPublic
Actions

Description

Hi,

Dayz 1.24 Stable server running on AlmaLinux release 8.7 (Stone Smilodon)

Every JSON file generated by any mod that uses it to store data gets corrupted overtime with extra characters, generating server crash.

Error example (Intenz-Leaderboard Mod Id 1758815806):

Unknown, 28.03 2024 11:41:37
[::JsonLoadFile] :: [ERROR] :: Cannot load data from "$profile:Leaderboard\Stats-76561199375471225.json":
JSON ERROR:
The document root must not be followed by other values.
Function: 'JsonLoadFile'
Stack trace:
scripts/3_Game/tools\jsonfileloader.c:131
Leaderboard_Scripts/4_World/statlogger\statlogger.c:128
Leaderboard_Scripts/4_World/statlogger\statlogger.c:13
Leaderboard_Scripts/4_World/entities\manbase\playerbase.c:34
scripts/5_Mission/mission\missionserver.c:378
JM/CF/Scripts/5_Mission/communityframework\mission\missionserver.c:63
scripts/5_Mission/mission\missionserver.c:298
JM/CF/Scripts/5_Mission/communityframework\mission\missionserver.c:10
$CurrentDir:mpmissions/RatHouse.chernarusplus/init.c:1006
scripts/3_Game/dayzgame.c:1699
JM/CF/Scripts/3_Game/communityframework\game\dayzgame.c:97

Runtime mode
CLI params: config serverDZ.cfg port 2302 mod 1559212036;3186006767; serverMod IntenzLB; BEpath battleye profiles profiles dologs adminlog netlog freezecheck

Checking JSON we see it has extra characters and has become invalid:

Original JSON:

{

"deaths": [],
"kills": [],
"animalsKilled": [],
"name": "RoadRat",
"lastTimeSeen": "2024-3-28 12:48:51",
"deathsToZCount": 0,
"deathsToNaturalCauseCount": 0,
"deathsToPlayerCount": 0,
"deathsToAnimalCount": 0,
"suicideCount": 0,
"longestShot": 0,
"zKilled": 75,
"timeSurvived": 22613,
"distTrav": 41375

}

Corrupted JSON:

{

"deaths": [],
"kills": [],
"animalsKilled": [],
"name": "RoadRat",
"lastTimeSeen": "2024-3-28 12:48:51",
"deathsToZCount": 0,
"deathsToNaturalCauseCount": 0,
"deathsToPlayerCount": 0,
"deathsToAnimalCount": 0,
"suicideCount": 0,
"longestShot": 0,
"zKilled": 75,
"timeSurvived": 22613,
"distTrav": 41375

}413

This in an example, but it happens with every mod using JsonWriteFile fuction.

Mods tested with same problem:

  • Intenz-Leaderboard 1758815806
  • Harks Stalker Leader Board 3017442961
  • VPP Admin Tools 1828439124
  • VanillaPlusPlusMap 1623711988

And every single mod using JSON to store and retrieve data I have tried.

Is a Linux issue as all this mods are working properly under Windows (tested).

At this point, I am completely lost. Can you help me?

I initially thought could be something related to my specific Linux file system, but it's a dedicated server running for months a lot of enterprise applications with no errors at all. So I assume it must be related to JsonWriteFile function.

Did you have any similar reports?

Cheers,
Micky.

Details

Severity
None
Resolution
Open
Reproducibility
N/A
Operating System
Linux x64
Operating System Version
AlmaLinux release 8.7 (Stone Smilodon)
Category
General
Steps To Reproduce
  1. Set up a dedicated AlmaLinux release 8.7 server.
  2. Install Dayz 1.24 Stable for Linux.
  3. Install any mod using JSON to write and read game data, like a Leaderboard mod, by example.
  4. Run server, Error will appear after some minutes, causing server crash.

Event Timeline

RoadRat created this task.Mar 28 2024, 1:35 PM
Geez changed the task status from New to Assigned.Mar 28 2024, 2:48 PM
Geez assigned this task to dedmen.
Geez added subscribers: dedmen, Geez.

@dedmen

WoozyMasta added a comment.EditedJun 20 2024, 2:20 AM

I also encountered this problem, when writing json, it seems that line by line writing to the file occurs, but the file is not cleared before writing.

Something like this:

  • File state before writing:
{
"a": 1,
"b": 2
}
  • After rewriting the document with the content:
{
"с": 3
}
  • I get something like this:
{
"с": 3
}
} // I think this line is from the previous state

Sometimes you get a completely valid document, but with completely wrong data, and sometimes it breaks syntactically. I think you can easily reproduce this if you write different test data structures each time

WoozyMasta added a comment.Jul 5 2024, 4:34 PM

https://steamcommunity.com/sharedfiles/filedetails/?id=2905487932
https://steamcommunity.com/sharedfiles/filedetails/?id=2731250878
These mods will definitely show the problem, cut down a tree or use a gas station, stop the server after a short time (so that the files are updated) and the files will be broken

@dedmen

dedmen added a comment.Jul 12 2024, 5:27 PM

This is such a pain to reproduce.
https://steamcommunity.com/sharedfiles/filedetails/?id=2905487932
I tried it with this mod.

Start server, cut couple trees, stop server, start server, cut couple trees, stop server. No issues, everything seems good.

I CAN see how the corruption COULD happen. But it just doesn't make sense.
In JsonSerializer.WriteToString()

There is indeed leftover data in the buffer after it serializes the data into memory.

This is the buffer after a write.

{
    "Roles": [
        "everyone"
    ]
}munityframework\logging\cf_trace.c : 55
CF_Trace_1() JM/CF/Scripts/1_Core/communityframework\logging\cf_trace.c : 226
Load() JM/COT/Scripts/4_World/communityonlinetools\classes\permissionsold\jmplayerinstance.c : 494
OnCl!

But when it grabs the memory buffer, and turns it into a string to return, the code pushes a null character onto where the end is supposed to be.
And....

{
    "Roles": [
        "everyone"
    ]
}

It ends up returning this. Which is perfectly fine.

I have also been running Community Online Tools on my server for over a month now.
It rewrites its PermissionFramework/Players json files regularly, basically on every join/leave. And I never observed issues.
And also now, I cannot produce any problems with either COT or the persistent tree thing.

And this code hasn't been touched since mid 2022, so nothing was fixed since then

dedmen added a comment.Jul 12 2024, 6:12 PM

In the end it wasn't json at all.
It was writing to files.
Here is repro code

void FileTest1()
{
	FileHandle handle = OpenFile("$profile:\\test.txt", FileMode.WRITE);
	FPrint(handle, "ABCDEFGHIJK");
	CloseFile(handle);
}

void FileTest2()
{
	FileHandle handle = OpenFile("$profile:\\test.txt", FileMode.WRITE);
	FPrint(handle, "X");
	CloseFile(handle);
}

void main()
{
	FileTest1();
	FileTest2();
}

Ends up with "XBCDEFGHIJK"

dedmen changed the task status from Assigned to Feedback.EditedJul 12 2024, 6:25 PM

Fixed 1.26

© Bohemia Interactive a.s. Bohemia Interactive® is a registered trademark of Bohemia Interactive a.s. All rights reserved. · Privacy Policy · Terms and Conditions