Same for Arma 3 dedicated Linux server with 4-10 players:
Server keeps running, Clients crash (all clients connected to that single server at once).
Happens regularly every 20-30 min.

Thanks k0rd for pointing me to that forum entry. However, I'm, already using a solution which is far more easier and achieves the same, c.f. my last comment at http://feedback.arma3.com/view.php?id=18487#c70219

The reason I want to use the Intel TBB allocators instead of glibc's malloc + nop-mallinfo is another 10% performance boost I observed on our server with our mission.

You still need to inject. However, 1.18 broke it for me due to http://feedback.arma3.com/view.php?id=18706

I don't have a working Intel TBB solution for 1.18 at this time, but I'm working on it.

Btw, the dedicated server part doesn't seem to have any anti-cheat protections, at least I didn't run into any problems.

In case you keep debugging symbols for your released binaries somewhere:
For the 1.18 binary, the uninitialized pthread_key_t variable is located at 0x09b0a5a0.

For the case you haven't, I'll give you some more details to locate the bug in your sources:
I've looked a second time at this and actually, there _is_ an initialization function called at program startup: It is located at 0x08154e60.
It is called from the function at 0x0805e270 which is stored in the .init_array (ELF-)section and thus is called at program initialization time.

It looks like 0x08154e60 is the constructor of some C++ object with static storage duration (not necessarily declared with the 'static'-keyword).
That object wraps a pthread_key_t member.
[That 0x0805e270 stored in the .init_array section is certainly a compiler generated helper function calling all your static storage duration objects' constructors.]

This constructor at 0x08154e60 even calls pthread_key_create(), but it throws the result away and stores a hardcoded zero in its pthread_key_t member.
This is the bug.

So go, grep for "pthread_key_create" in your sources, pick the one called from a C++ object's constructor and check that constructor for the bug described above.
If there is more than one such class, choose the one which is instantiated with static storage duration, i.e. for which there is a global/static variable of that class' type.

Should be a 10min fix, right? To be honest, I've got no idea why this issue stays in the "new"-state for nearly two weeks now. At least someone could have hit the magic "Reviewed"-button in the meanwhile (or asked a question if something is unclear).

LD_PRELOADing is a (unixish) technique to change a program's behaviour without its explicit support, i.e. without changing its source (see 'man ld.so' on any Linux box).

It would be a very cheap way of actually supporting custom memory allocators without any coding.

As I wrote above, LD_PRELOADing is not working due to a bug unrelated to LD_PRELOADing at a first glance (usage of unitialized static variables).

To give you guys some more input:

I did succeed to fix the bug described above by manually injecting some machine code doing the missing initalization into the 'arma3server' binary and LD_PRELOAD works for me now.

My benchmark mission is a MP mission using the @ALIVE mod to automatically spawn some AI. My test system is a remote one, being equipped with an Intel Xeon E3 with TurboBoost enabled.

Results (FPS had been measured with '#monitor':

• Without Intel TBB allocator LD_PRELOADed:
• The mission, especially @ALIVE, takes incredibly long to load (up to 15 min).
• FPS are very unstable.
• FPS with only 35 AI spawned is 3 to 15, 3 far more often.

• With Intel TBB allocator LD_PRELOADed:
• The mission, including @ALIVE, loads fast. Although not measured explicitly, it seems to load faster than with the dedicated server for MS Windows operating under Wine.
• FPS is stable.
• FPS with 243 AI spawned is 35-45. Even at spawning, the FPS doesn't go below 30. With only 35 AI spawned, it is constantly at 49.

I suspect that the gain in performance is not so much related to the exact memory allocation implementation, but to the fact that Intel TBB's 'mallinfo()' implementation is a nop, i.e. it always returns zero. C.f. the issue I opened at http://feedback.arma3.com/view.php?id=18487

I suspect that any LD_PRELOADing will segfault arma3server since the real reason seems to be that the dl-framework is entered early.
However, I observed that issue with the Intel TBB allocators.

server version string seems to be "1.16.113494"

FIXED
Thank you.

Since people start compiling their hand-tweaked LFS-glibc already (c.f. http://forums.bistudio.com/showthread.php?169926-Linux-Dedicated-Server-feedback/page18), I'll give you my far more easier solution which will work on any Linux and without glibc modifications:

• Get the attached dummy_mallinfo.c.txt and rename it to dummy_mallinfo.c.
• Turn it into a shared object by
• gcc -shared -m32 -fPIC -o dummy_mallinfo.so dummy_mallinfo.c
• Set the environment variable LD_LIBRARY_PATH to the _absolute_ directory containing that new dummy_mallinfo.so:
• export LD_LIBRARY_PATH=/path/to/wherever/:"$LD_LIBRARY_PATH"
• run arma3server by
• LD_PRELOAD=dummy_mallinfo.so ./arma3server

This will replace glibc's costly mallinfo() with my nop-implementation at runtime.

dazhbog: Glad this one got assingned.

Unfortunately, I can't link issues here, so be sure to have a look at
http://feedback.arma3.com/view.php?id=18556 which is somewhat related.

I did another test:
in order to exclude 32bit-related integer overflow issues, I rerun the above test in a virtual 32bit Ubuntu 12.04 (i386) installation with only 2047MB of RAM attached.

Command line of 'arma3server':

./arma3server -port=2302 -config=serverconfig/server.cfg -name=foo -noSound -world=empty

Performance sampling commands:

1. sudo perf record -p <arma3server_pid> sleep 10
2. sudo perf report > perf_report_arma3server_virt_ubuntu1204_i386_2047MB.txt

Again, the same results: dominant activity in glibc's 'mallinfo()'.
I'll attach the output file 'perf_report_arma3server_virt_ubuntu1204_i386_2047MB.txt' to this issue.

btw, server version string seems to be "1.16.113494"