Page MenuHomeFeedback Tracker

FileSerializer - FileMode.APPEND without function since 1.27
Confirmed Internally, NormalPublic

Description

Running

FileSerializer s = new FileSerializer();
if (s.Open("$profile:test4.bin", FileMode.WRITE)) {
    Print("DEBUG WRITE - TEST")
    s.Write("TEST");
    s.Close()
} else {
    Print("DEBUG FAILED TO OPEN FOR WRITE")
}
for (int i = 0; i < 2; i++)
{
    if (s.Open("$profile:test4.bin", FileMode.APPEND)) {
        Print("DEBUG APPEND - TEST")
        s.Write("TEST APPEND");
        s.Close();
    } else {
    Print("DEBUG FAILED TO OPEN FOR APPEND")
    }
}

on top of void main() in init.c gives

SCRIPT       : DEBUG WRITE - TEST
SCRIPT       : DEBUG APPEND - TEST
SCRIPT       : DEBUG APPEND - TEST

in the log (so opening the file doesn't fail), but even after a restart the content of test4.bin is only

00000000: 0400 0000 5445 5354                      ....TEST

Append doesn't have a function at all on Linux currently.
My impression is that it got worst since the 1.27 bugfix, but I can't prove that.

https://github.com/Arkensor/DayZ-CommunityFramework/issues/183 is the result of this issue, maybe also https://feedback.bistudio.com/T189830

Details

Severity
Block
Resolution
Open
Reproducibility
Always
Operating System
Linux x64
Operating System Version
Debian Bookworm
Category
Scripting
Steps To Reproduce

See above

Event Timeline

bzed created this task.Wed, Mar 19, 5:58 PM
lava76 added a subscriber: lava76.Wed, Mar 19, 10:44 PM
Geez changed the task status from New to Assigned.Thu, Mar 20, 11:39 AM
bzed added a subscriber: Geez.Fri, Mar 21, 1:41 PM

@Geez More testing (thanks to @lava76 ) shows that append is also broken for a normal FileHandle.
Also a plain vanilla chernarus seems to have issues with persistance (things don't despawn in time and similar things), so my impression is its broken for the server in general. This is a really major issue, please fix it asap!

bzed added a comment.Fri, Mar 21, 1:54 PM

So with the following code

FileHandle handle;
for (int i = 0; i < 2; i++)
{
    handle = OpenFile("$profile:testraw.bin", FileMode.APPEND);
    if (handle) {
        Print("DEBUG APPEND - TEST");
        FPrint(handle, "TEST APPEND");
        CloseFile(handle);
    } else {
        Print("DEBUG FAILED TO OPEN FOR APPEND");
    }
}

strace shows

472   openat(AT_FDCWD, "/profiles/testraw.bin", O_RDONLY|O_CREAT|O_APPEND, 0660) = 190
472   stat("/profiles/testraw.bin", {st_dev=makedev(0xfd, 0x2), st_ino=6567647, st_mode=S_IFREG|0640, st_nlink=1, st_uid=0, st_gid=0, st_blksize=4096, st_blocks=0, st_size=0, st_atime=1742560455 /* 2025-03-21T12:34:15.408050679+0000 */, st_atime_nsec=408050679, st_mtime=1742560455 /* 2025-03-21T12:34:15.408050679+0000 */, st_mtime_nsec=408050679, st_ctime=1742560455 /* 2025-03-21T12:34:15.408050679+0000 */, st_ctime_nsec=408050679}) = 0
472   write(172, "SCRIPT       : DEBUG APPEND - TEST\n", 35) = 35
472   write(190, "TEST APPEND", 11)     = -1 EBADF (Bad file descriptor)
472   fcntl(190, F_GETFL)               = 0x8400 (flags O_RDONLY|O_APPEND|O_LARGEFILE)
472   close(190)                        = 0
472   openat(AT_FDCWD, "/profiles/testraw.bin", O_RDONLY|O_CREAT|O_APPEND, 0660) = 190
472   stat("/profiles/testraw.bin", {st_dev=makedev(0xfd, 0x2), st_ino=6567647, st_mode=S_IFREG|0640, st_nlink=1, st_uid=0, st_gid=0, st_blksize=4096, st_blocks=0, st_size=0, st_atime=1742560455 /* 2025-03-21T12:34:15.408050679+0000 */, st_atime_nsec=408050679, st_mtime=1742560455 /* 2025-03-21T12:34:15.408050679+0000 */, st_mtime_nsec=408050679, st_ctime=1742560455 /* 2025-03-21T12:34:15.408050679+0000 */, st_ctime_nsec=408050679}) = 0
472   write(172, "SCRIPT       : DEBUG APPEND - TEST\n", 35) = 35
472   write(190, "TEST APPEND", 11 <unfinished ...>
499   <... clock_nanosleep resumed>NULL) = 0
472   <... write resumed>)              = -1 EBADF (Bad file descriptor)
499   clock_nanosleep(CLOCK_REALTIME, 0, {tv_sec=0, tv_nsec=10000000},  <unfinished ...>
472   fcntl(190, F_GETFL)               = 0x8400 (flags O_RDONLY|O_APPEND|O_LARGEFILE)
472   close(190)                        = 0

you are mixing O_RDONLY and O_APPEND, use O_RDWR or O_WRONLY.

Corse added a subscriber: Corse.Sun, Mar 23, 7:48 AM
bzed added a comment.EditedMon, Mar 24, 11:59 AM

@Geez I don't want to put pressure on you - but currently every linux server is broken due to that. When are you going to fix it?

bzed added a comment.Mon, Mar 24, 8:41 PM

https://github.com/bzed/dayz-append-fix/ - here is a workaround until its fixed properly.

SirAdef added a comment.EditedMon, Mar 24, 9:33 PM

https://github.com/bzed/dayz-append-fix/ - here is a workaround until its fixed properly.

Some workaround has been added to CF too

  • Added a workaround for player loading under Linux due to DayZ bug T190201 (FileMode.APPEND without function)
bzed added a comment.Mon, Mar 24, 9:36 PM

https://github.com/bzed/dayz-append-fix/ - here is a workaround until its fixed properly.

Some workaround has been added to CF too

  • Added a workaround for player loading under Linux due to DayZ bug T190201 (FileMode.APPEND without function)

Yes, that is a crude hack though and doesn't fix the issue properly - its impossible to do that within a mod.
You'll realize that with my LD_PRELOAD you'll suddenly have logs from various mods that were empty before.

That bug must have been lingering longer, I guess it only affected "normal" file writes before and made a mess as soon as the Filestream used by CF was affected.

Corse added a comment.Tue, Mar 25, 2:53 AM

https://github.com/bzed/dayz-append-fix/ - here is a workaround until its fixed properly.

Some workaround has been added to CF too

  • Added a workaround for player loading under Linux due to DayZ bug T190201 (FileMode.APPEND without function)

Yes, that is a crude hack though and doesn't fix the issue properly - its impossible to do that within a mod.
You'll realize that with my LD_PRELOAD you'll suddenly have logs from various mods that were empty before.

That bug must have been lingering longer, I guess it only affected "normal" file writes before and made a mess as soon as the Filestream used by CF was affected.

You're a gentleman and a scholar, sir. I will give this a try tonight and report back.

Server running all night no problems... Next days will show if it really is stable enough.

For now, another point in yeold match Devs vs. Community...

Corse added a comment.Tue, Mar 25, 7:53 AM

Server running all night no problems... Next days will show if it really is stable enough.

For now, another point in yeold match Devs vs. Community...

I was able to throw in one of my old loadouts with my init.c file and then loaded this up before my startup script and so far it's working fine. No issues thus far. CentOS 9 here.
Same though, I will wait a few days to see if anyone reports issues. It's been very hit or miss. Sometimes it'll be ok for a few hours, other times not. But if it holds up a few days we'll see. Fingers crossed.

bzed added a comment.Tue, Mar 25, 8:02 AM

With the workaround in CF in place it's not super critical anymore, at least if you don't have other mods that append data to Filestreams.
You can check your log, stderr should have a line for each modified function call.

Corse added a comment.Tue, Mar 25, 9:23 AM

With the workaround in CF in place it's not super critical anymore, at least if you don't have other mods that append data to Filestreams.
You can check your log, stderr should have a line for each modified function call.

I wonder if this is part of the issue that has plagued the banking mods on linux (and others...). I know some of them were made off of each other but it seems on linux that randomly, they would incorrectly save files and files would go corrupt and players would report issues with their bank. You could go in and check everything and sure enough there was incorrect data that would make invalid json formatting. I thought, "That is really weird. Either something isn't appending stuff correctly, or line characters are wrong, or something is writing to these super weird and it corrupts on restart. Why linux and not windows?" I got around it by making the core config file immutable by root so the normal user couldn't edit it by the mod anymore and suddenly no more corrupt file issues LOL. I'll have to test it out now that the workaround is in place.

bzed added a comment.EditedTue, Mar 25, 10:11 AM

@Corse I only know the expansion atm mod, that's writing JSON files with a single write, that always worked. All mods that append data were broken, including basic things like log files. They were always empty before, so I think writes for simple files was broken since a longer time.

bzed added a comment.Tue, Mar 25, 10:15 AM

@Geez fyi: Append is broken for File and FileStream

bzed added a comment.Tue, Mar 25, 12:27 PM

https://github.com/bzed/dayz-append-fix/ - here is a workaround until its fixed properly.

I've commented out some unnecessary code. The only function that seems to be affected is open, you can reduce the lib more if you want, but it also shouldn't make a difference.

Geez changed the task status from Assigned to Confirmed Internally.Tue, Mar 25, 4:49 PM

A fix for this is being tested internally. Thank you, everyone, for your input.

@Corse I only know the expansion atm mod, that's writing JSON files with a single write, that always worked. All mods that append data were broken, including basic things like log files. They were always empty before, so I think writes for simple files was broken since a longer time.

Been using your workaround for a few days now and so far it's been working absolutely perfect. So far things are perfectly stable.

bzed added a comment.Wed, Mar 26, 11:37 PM

Glad to hear @Corse!

bzed added a comment.Sun, Mar 30, 2:10 PM

I gave 1.27 from experimental a quick try and it seems this issue is solved there already. Thanks for the quick fix!