Hello,

We need the .mdmp files located in the same folder as .bidmp and .rpt files.

Thank you.

mdmp files are never created.

I have yet to see an mdmp file produced by any of these crashes, either in the profile folder, nor have any OS dumps been created by my user-mode dumps being enabled.

It seems you have custom memory allocator running. This can cause stability issues with your game. Please try disabling it and see if the issue persists.

Ok - turned on default (and inferior :P) allocator, here is the latest RPT and bidmp file. Again, no mdmp file is being created. Also, no user-mode dump.

https://paronity.com/i/crash5_default_malloc.zip

Got a crash totay that actually seems to have been caught "correctly". There was an mdmp file in it this time and the RPT reports an access violation.

Here is the zip with RPT, BIDMP, and MDMP: http://paronity.com/i/dump_default_malloc_with_mdmp.zip

I have no idea what your MDMP file contains, so just in case it's helpful, the user-mode dump ALSO threw and I have a full dump should it be necessary. Also, just in case, here is the WinDBG anaylyze output for starters.I know details are scarce without symbols, but I figured I'd throw it in just in case. :)

• *
• Exception Analysis *
• *

• ERROR: Symbol file could not be found. Defaulted to export symbols for tbb4malloc_bi.dll -
• ERROR: Symbol file could not be found. Defaulted to export symbols for PhysX3Common_x86.dll -
• ERROR: Symbol file could not be found. Defaulted to export symbols for tier0_s.dll -
• ERROR: Symbol file could not be found. Defaulted to export symbols for steamclient.dll -
• WARNING: Unable to verify checksum for extdb.dll
• ERROR: Symbol file could not be found. Defaulted to export symbols for extdb.dll -

FAULTING_IP:
arma3server+db603a
00ee603a f3a4 rep movs byte ptr es:[edi],byte ptr [esi]

EXCEPTION_RECORD: ffffffff -- (.exr 0xffffffffffffffff)
ExceptionAddress: 00ee603a (arma3server+0x00db603a)

 ExceptionCode: c0000005 (Access violation)
ExceptionFlags: 00000000

NumberParameters: 2

Parameter[0]: 00000000
Parameter[1]: 00000000

Attempt to read from address 00000000

CONTEXT: 00000000 -- (.cxr 0x0;r)
eax=00000000 ebx=00000000 ecx=00000000 edx=00000000 esi=00000003 edi=00000003
eip=7744d2ec esp=01bea95c ebp=01beaadc iopl=0 nv up ei pl nz na pe nc
cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00200206
ntdll!NtWaitForMultipleObjects+0xc:
7744d2ec c21400 ret 14h

DEFAULT_BUCKET_ID: NULL_POINTER_READ

PROCESS_NAME: arma3server.exe

ERROR_CODE: (NTSTATUS) 0xc0000005 - The instruction at 0x%08lx referenced memory at 0x%08lx. The memory could not be %s.

EXCEPTION_CODE: (NTSTATUS) 0xc0000005 - The instruction at 0x%08lx referenced memory at 0x%08lx. The memory could not be %s.

EXCEPTION_PARAMETER1: 00000000

EXCEPTION_PARAMETER2: 00000000

READ_ADDRESS: 00000000

FOLLOWUP_IP:
arma3server+db603a
00ee603a f3a4 rep movs byte ptr es:[edi],byte ptr [esi]

NTGLOBALFLAG: 0

APPLICATION_VERIFIER_FLAGS: 0

APP: arma3server.exe

ANALYSIS_VERSION: 6.3.9600.17298 (debuggers(dbg).141024-1500) x86fre

FAULTING_THREAD: 00000f70

PRIMARY_PROBLEM_CLASS: NULL_POINTER_READ

BUGCHECK_STR: APPLICATION_FAULT_NULL_POINTER_READ

LAST_CONTROL_TRANSFER: from 00e96585 to 00ee603a

STACK_TEXT:

STACK_COMMAND: ~0s; .ecxr ; kb

SYMBOL_STACK_INDEX: 0

SYMBOL_NAME: arma3server+db603a

FOLLOWUP_NAME: MachineOwner

MODULE_NAME: arma3server

IMAGE_NAME: arma3server.exe

DEBUG_FLR_IMAGE_TIMESTAMP: 55253f19

FAILURE_BUCKET_ID: NULL_POINTER_READ_c0000005_arma3server.exe!Unknown

BUCKET_ID: APPLICATION_FAULT_NULL_POINTER_READ_arma3server+db603a

ANALYSIS_SOURCE: UM

FAILURE_ID_HASH_STRING: um:null_pointer_read_c0000005_arma3server.exe!unknown

FAILURE_ID_HASH: {785b27d8-09bd-7a57-4840-60685f9f0af0}

Followup: MachineOwner

If you need the full dump uploaded, just let me know.

Another one. No MDMP or user-dump on this one:

https://paronity.com/i/crash_6_default_malloc.zip

Again,

https://paronity.com/i/crash_7_default_malloc.zip

No MDMP or User-mode.

How long should I expect before I get some sort of movement on this? Should I keep uploading these files? Are they helpful at all? Any tips on what I can do to expedite the process would be great. Thanks!

Hello, if the server crashes again with .rpt .mdmp .bidmp files all being created please upload them.

Thank you.

It has crashed several times, and the frequency seems to be more after the last push this week. It's never throwing an mdmp file.

We did finally get one today that had all three. It's attached and here as well: https://paronity.com/i/crash_9_default_malloc.zip (RPT, MDMP, and BIDMP)

Is there anything else we need to make some progress on this one? Is there anything to note about it only giving MDMP files like 5% of the time? Anything I can do to "influence" them to be caught correctly? I know that exceptions (especially memory), are a slippery slope, just trying to make progress. :)

I've had many server crashes since 1.42. 1.44 is on a different level though. I get sometimes 3 crashes a day. I'm adding my crash logs to this tracker.

crash_logs.7z

Thank you for attaching the new dump files. We will take a look at them.

Hello, please try using the profiling branch and see if that fixes your issues. Thank you.

Still seeing the crashes on my end. MDMP files are not being created still, either....

Is it still crashing even after the 1.46 update?

Yup. Twice yesterday, and once today so far.

As the crash does not generate MDMP dump files it is most probably either Memory or Windows related crash.

It's not windows. If it were windows, there would be events being thrown and a memory dump would be created (because I have user-mode dumps enabled for that executable).

Memory related crash is possible, but you, as the software that is running are responsible for managing those memory calls and handling them should they fail.

Is there anything I can do to help further debug the issue on your end? I'm an operating system level debugger for a living and have a lot of experience in this area, I just need to know what I can do with your software to do so. I am doing everything possible from the OS level, and am getting no where, which leads me to believe it's on ARMA side of things.

Are you by any chance running any mods?

Just extDB for database connectivity.

Bump.....

So there is nothing more that I can do to debug this issue?

The reason why mdmps are not generated might be due to the server running out of memory.

The only thing on the box is this particular server (and the MySQL instance for it) - and the server has 32 GB of RAM.

Most of the time, the ARMA server idles around 110-200 MB of RAM (with the 3rd party malloc) and 1.4GB-1.8GB (with the default malloc) depending on player count, object count, ect....

The system NEVER gets above 25% RAM usage (if we are doing testing with another server, or something else with the box at that point in time). Point being, we aren't even coming close to running out of RAM.

https://paronity.com/i/8i8I3.png

Could you please try reproducing the issue on Vanilla version of the game and Running Vanilla maps?

Altis is a vanilla map......

Your software isn't handling an exception the way it should. That is a fact. If it were, we would be getting MDMP files. It's irrelevant if I'm running a mod or not.

Just close this issue out. It's apparent that it's going no where. Thanks.

But you are using community made mission (Altis life) which very well could be the reason why you are experiencing this issue.

99,9% of the missions played in MP are community made missions. If they cause crashes then you should find out why your game crashes (it's still the game which crashes not the community made mission).

How should the mission maker find out what causes the crash without having access to the engine?

Thanks Xeno - my points exactly.

If you create an engine that is flexible and made for customization, then you need to handle exceptions correctly so that the modders knows what went wrong and how to fix it.

I am not disagreeing that the code written could be causing the issue, but how am I to determine what code is causing the issue when your engine isn't handling the exception correctly?

I'm just asking that you try running a MP map that is officially made by Bohemia.

I don't consider that a valid debugging step and I will not be able to test that, and additionally, of course it's going to work.

I can even run the Altis Life file just fine with no one on it and it runs just fine and never crashes.

For the sake of proving absolutely nothing, I ran a server on the box (default map with no customization) since my last post, with a tool restarting it every 12 hours and it never crashed. So, as was already obvious it is something in the custom code that is causing the issue (but we already knew that), but that doesn't help us (content creators and server admins) figure out what is causing the issue because it's YOUR engine that isn't handling the exception correctly. What's more, your code is "handling" it enough to prevent windows from seeing the failure (hence no user-mode dumps and no event log entries) but not enough to do its job.

I would like to be able to figure out what code is causing the issue, but you, the creator of the engine, provide me no valid way of doing so.

I poked around in the hex editor in the bidmp files and can see the error
that it "thinks" is causing the issue which is:
"Out of memory (requested 4203 KB).

footprint 536870912 KB.
pages 81920 KB.

B, mapped 50040832 B), free 60440576 B", but as I said before, the server is on a box with 32 GB of RAM (https://paronity.com/i/6Z7g3.png) and its the only thing running. It's RAM consumption stays around the 110MB-169MB range. (https://paronity.com/i/1E2l4.png)

Altis Life is running a Multiplayer Map that is officially made by Bohemia, it resides on the Altis map.

The content in Altis Life is created with the SQF language, Bohemia's official language for the Arma series. This language is maintained & surrounded by your container - the game.

Closing as expired. If the issue persists please create a new ticket. Thank you.