Page MenuHomePhabricator

Server crashes randomly
Assigned, WishlistPublic

Description

Since the last arma patch our altis server has been crashing randomly with access violations.

It runs fine during night when it's not that much player online, but during the day it crashes every 2 hours or so. Worked perfectly before the patch... {F27523} {F27524} {F27525} {F27526} {F27527} {F27528} {F27529} {F27530} {F27531} {F27532} {F27533}

Details

Legacy ID
2929659189
Severity
None
Resolution
Open
Reproducibility
Random
Category
Dedicated Server
Steps To Reproduce

Tried every function in-game, vehicles etc. Was not able to reproduce, crashing seems random. I tried reinstall arma server, reinstalled redist and still does not work.

Additional Information

I provided with rpt log and bi dump log. Ignore that housing error expression.

RPT LOGS
http://s000.tinyupload.com/index.php?file_id=34019274144763348848

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

We are still getting crashes on newest hotfix, as of 12/17/2015 11:30pM EST
Using TBB4malloc, memory error, it was not an .exe error this time
http://i.imgur.com/AEXeCJO.png

Player Count was 60/100

Dwarden released new binaries just as we speak

1.54.133788 profiling & performance v6 server and client, windows, linux server too
+ several more crash cases were fixed since v5
https://www.dropbox.com/sh/582opsto4mmr8d8/3BSy9PdRGm

We Crashed again too on Vers. 1.54.133741

On every Crash the TaskManager shows exactly 2.000.000 K Ram .. Maybe other Admins can confirm my observation.

https://www.dropbox.com/s/0qw4u3wa785o64d/DieLiga_ServerCrash_2.zip?dl=0

Hey Everyone.

I have been doing quite a bit of testing with this issue, and so everyone is aware I host OzzyGaming.com Altis Life server. We have been plagued with these crashes. On average from 10 minutes to an hour at best.

I have done quite abit of research into what has been happening and One thing stands out, Windows.

As such I wanted to confirm my findings by setting up a server hosted under linux. So I purchased a VPS with Ubuntu pre-installed and installed my server on that. And the results were shocking to say the least.

Not one crash at all. Desync just as you'd expect from a low grade VPS server but that server survived without a crash for upto 7 hours where it was restarted normally.

This issue has been around since late July. Not sure why but there was evidence of this issue back then. So Bohemia Interactive this has something to do with the way ARMA is trying to store data to memory under a Windows Based environment.

Also anyone experiencing this issue. grab a VPS server with what you afford and or use HyperV to run ARMA under Ubuntu or other linux based OS.

Uploaded crashes with performance v6 (Crash_withV6.rar).

CHIBANE added a subscriber: CHIBANE.May 8 2016, 1:14 PM

Hello ! wake up always crashes servers and you make no communication wake up !! Thank you

@CHIBANE

Most of them have gone to christmas holidays, give them some time. This is their work and they have real family too :)

@rasmusop

Yes you are right but the problem is the December 14 and no news ..

They are working on it, as they have provided us with 7 performance builds. There's not only one crash issues, there's many.

raingod added a subscriber: raingod.May 8 2016, 1:14 PM

We have tested all patches in drop box and still crashing!

i have the same problem too

PLEASE Bohemia can you found a Solution for that problem

PLEASE Bohemia can you found a Solution for that problem

PLEASE Bohemia can you found a Solution for that problem

PLEASE Bohemia can you found a Solution for that problem

PLEASE Bohemia can you found a Solution for that problem

PLEASE Bohemia can you found a Solution for that problem

PLEASE Bohemia can you found a Solution for that problem

PLEASE Bohemia can you found a Solution for that problem

PLEASE Bohemia can you found a Solution for that problem

PLEASE Bohemia can you found a Solution for that problem

PLEASE Bohemia can you found a Solution for that problem

PLEASE Bohemia can you found a Solution for that problemPLEASE Bohemia can you found a Solution for that problemPLEASE Bohemia can you found a Solution for that problemPLEASE Bohemia can you found a Solution for that problemv

PLEASE Bohemia can you found a Solution for that problem

@Preda Spamming isn't going to speed anything up. I'm sure BI understands that this is a high priority job.

@Preda i hope you get banned for this shit spam

We waiting a solution since 14 december 2015.
Sleeping or what? This is urgent fucking shit!

Behemia where is your support? Stop dev "End Game" and "Tanoa".
Give us a solution to this problem.

Thank you in advance for an answer Bohemia !

tim roth has right

@timrothfr. There is a workaround.

I have already stated twenty times already.

ARMA 3 Servers are only crashing under the Windows based environment.

Grab another server (VPS as this is only a temporary measure) and have it running Ubuntu. Install ARMA 3 server on that and you won't have a crash.

the server cost a lot of money if we change for linux we need a solution by bohemia

Server Crash again on V6 ... Server full with 75 player , Runtime 6 H > no Crash .. Server empty with ~ 35 Player ,Runtime 2 H > Crash ...

??????
http://www.bilder-upload.eu/show.php?file=232ed2-1451396858.png
https://www.dropbox.com/s/lo6i55kho4i134f/DieLiga_ServerCrash_6.zip?dl=0

Can´t get the Performance from Linux V-Box that I need .. So .. wan´t change the hole System to Linux for a "maybe" solution .. I think it´s also hard to get the Performance from the Linux machine or it take a lot of time to adjust the server to get the Performance ..

So .. still wait for a Solution from Bohemia ....

yes hirschi but bohemia prefer dev on End game and Tanoa and don't resolve the problem on altis and stratis

Where is Bohemia? To the beach.
Return to the updated 1.54.

and the nexte one on V6 ...Yeahhh .. 30 Player online .. 4 Hours
trying the V7 tomorrow ...
https://www.dropbox.com/s/k6tytswfxmyb750/DieLiga_ServerCrash_7.zip?dl=0

And every Time another Crash Error ..

Yes ... I confirm with u Preda

1.52 was stable ... good Performance ... Restart Times of 8 Hours was no Problem with full Server ... and now this ...

I don´t need Tanoa with that Problems ... Sry

tim roth we can't return to 1.54 because the players will need to downgrade the game to play on our server

Guys i have the same issue on my windows server please vote it up too http://feedback.arma3.com/view.php?id=27150

USE v7... crashes with v6 are not longer interessting, because many are fixed with v7...

Waiting for an answer by BOHEMIA.
The bohemia team is not very responsive.

Waiting for an answer by BOHEMIA.

we are the 5/01/2016 NO RESPONSE BY BOHEMIA

in 2016 bohemia will help us ?

Bohemia please ... WAKE UP !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

HEY WAKE UP BOHEMIA

Our Server is losing all the Players. Many players start play other Games now…
Do something bohemia! I uploaded new Crashes

We still have crashes too .. on v6 on v7 ... what ever .. I have uploaded every crash ..

In 2017 bohemia will perhaps help us

I don’t think Bohemia will do something… They don’t answer us, but there they do: http://feedback.arma3.com/view.php?id=27272
They destroyed my Favorite Game :´(

We have been getting these types of errors using V7 exe, tbb4malloc,

http://puu.sh/mpn1x/05c726d28a.png

Any suggestions, every crash ends with this error

Thank you in advance

parsec added a comment.Jan 9 2016, 5:31 PM

Our community is dead now since BI took WAY too long to fix the problem. We went from 30 to 50 active players to 2 now and both of us think about quitting arma for good.

CRASH AGAIN ! AGAIN ! AGAIN ! AGAIN !

Seriously Bohémia .. WAKE UP and : FIX THIS PROBLEM. last Official post Bohémia : 14th December 2015 .. SERIOUSLY ??????????????????????????????????

Yeah, where's the support here?
Not that I'm the only one with this issue (obviously) but I've just spent the last 7 months building a server I can't use since that update. Roll the update back to a stable one and test fixing this issue in your own model office environment. Not on our clients and servers...

Are they working on this or is it just not important enough? I mean 52 upvotes it not that much compared to 2k votes vor something like "Female soldiers models should be available in the game".

again on V_7.
V_8 is crashing after 3 Hours so we use the V_7 ...

https://www.dropbox.com/s/q7f1vlwqaqekfir/DieLiga_ServerCrash_13.zip?dl=0

@ BI .. are u need all the LOG´s or is it work for nothing ???
what u need, that we can Help u and us ??? Can we try some configs, allocator ???
or what kind of log´s u need ??
We also spend thousand of hours to develope our Mission .. Now we can´t do anything until u Fix the Problem ..

So please .. do something .. or let us know how we can help....

Again on V_7 .. This time "out of Memmory" Yeahhhh ...

Gentlemen, could we please all post the whole modlist that we are currently using to see if we can find any correlations?
mine would be:
-exile
-esseker
-allinpack(full)
-ryanzombies not active at the moment (causes crashes without end)
-infistar
-extDB

at time:
-Altis Life 3.1.4.8
-extDB2 (69)
-InfiStar

Also had Crashes with
-extDB (35)
-ASM

Fluit added a comment.Jan 12 2016, 7:39 AM

CBA A3
ACE3 also RHS optionals
ACRE2
Kunduz, Afghanistan
RHS: AFRF + USAF
Taliban Fighters
Sub Insurgents
All-in-arma Terrain Pack
3CB BAF equipment, units, weapons, vehicles

-ExtDB (35)
-life_server

Addons have nothing to do with the crash, because even a blank task crash the server.
Perhaps incompatible hardware (CPU, etc ..) or software.
You are all on Windows Server 2012 R2?

Windows Server 2016 Technical Preview 3 64-bit
Intel i7 @ 3.30GHz
8.00 GB RAM

Windows Server 2012 R2 64 Bit
64GB DDR5 RAM
Intel i7-5820k
SSD

Windows Server 2008 R2 64 Bit
32GB DDR3 RAM 2000mhz
i7 4790k @4.8 GHz
Arma Server on Ram Disk

perf v8 seems to fix the issue. Had the server running (only myself in it) for 4 hours, no errors listed at all.

Still keeping our server password on though for now while some systems still get tweaked but you guys should download it and see how it fares with 30+ players.

https://www.dropbox.com/sh/582opsto4mmr8d8/3BSy9PdRGm

Sakzy added subscribers: mod, Sakzy.May 8 2016, 1:15 PM
Sakzy added a comment.Jan 12 2016, 3:27 PM

Hi, i have the same problem ..

30players = full restart (4hours)
Less then 50 = 3hours then crash
90+ = 2/3 hours crash

@mod
@extDB2

Windows Server 2008 R2 64 Bit
Intel E5-1650v3 6c/12t 3,8 GHz
128 Go RAM DDR4 ECC 2133 MHz
2x480Go SSD SOFT

Grams added a comment.Jan 12 2016, 9:54 PM

Intel E5-2630v3
E5-2630v3
8c/16t

2,4 / 3,2 GHz

128 Go RAM

DDR4 ECC 1866 MHz

2x480Go SSD SOFT

Windows Server 2012 R2

@3LGStevo .. we still have crashes with PerfBin_V8 .. now we are back on V7 ..

V8 = Crash after 3 Hours
V7 = stable for feew Days but still crashing randomly

now we try other allocator with V7 ..

Many our Players have Problems with Game Crashes if the join on Server .. If they use the System allocators Game looks stable ...

Hmmm strange. Our server was online for a total of 12.5 hours yesterday before I took it down to test a mission file patch. There were still some object not found (message type_119/91) errors and after about 6 hours we got network message is pending...

Which Malloc files are you using with perf v8? I'm still on v7 perf malloc.
We haven't had any client crashes, but when we used perf v7 with v7 malloc we got crashes with less than 3 players after 10 minutes.

Sakzy added a comment.Jan 13 2016, 1:30 PM

I noticed that the V8 add 20 minutes before a server crash. ( tested with 80 players in game.)
So it's not really better...

With v8, it still crashes after 1.5 hours with 60 people.

befor we had used the tbb4malloc_bi_withSSE2_v4_4 .. With that we have some days two crashes and than feew days without crashes..( all with Perf V_/ )

Now we use V_7 with tbb4malloc_bi_withoutSSE2_v4_4 ... still no crashes for two day´s ..

Crash again ..

V_7
tbb4malloc_bi_withoutSSE2_v4_4

Bohemia sucks for fix the bug

Bohemia delete messages but do not answer our questions ^^

Use the v9 perf if you're having problems with v8

Sakzy added a comment.Jan 16 2016, 5:48 PM

With v9, it still crashes after 1.37 hours with 50 people.

The V7 is better.

We have had the same problem for about a month now too. Our server will randomly freeze or crash, sometimes a memory error other times no error at all. It looks like we are going over a month and it still hasn't been fixed

Crash Again ..

with V7 and tbb4malloc_bi_withoutSSE2_v4_4

also with System allocator ... with System crash after 1 Houre

now we are back on V7 with tbb4malloc_bi_withSSE2_v4_4 .. looks like the best for us ..

I confirm with u all ... it´s frustrating .. :-(

And 3 times again
V7 with tbb4malloc_bi_withSSE2_v4_4

uploaded reports ....

Grams added a comment.Jan 22 2016, 6:15 PM

I currently test the v11 (https://forums.bistudio.com/topic/160288-arma-3-stable-server-154-performance-binary-feedback/page-53?), has yet to see if the server crashes or not.

Bohémia a minimum should communicate on this issue, if only to tell us that work on it and that the publication of our logs are not useless!

V10 crash with 96players after 2hours.

New Error in RPTLog :
Streaming anim a3\anims_f\data\anim\sdr\mov\erc\stp\non\non\amovpercmstpsnonwnondnon_awoppercmstpsoptwbindnon.rtm
Anim stream offset 0 = 2566

They work on the Problem ....

https://forums.bistudio.com/topic/160288-arma-3-stable-server-154-performance-binary-feedback/page-53

They add some extra logs in the new Pref Bin to find the Issus ... They still wait for log´s from V 11 ... So maybe u publish u logs in the thread

Better is to send the dropbox link of your crash archive per skype to dwarden... all informations are in the forum... search in arma3 forum for skype groups an join decidated server group, all infos problems and other stuff is here first...

v11 crash added.

Using v11... entire animation system completely crashed.
Could no longer perform any kind of restrain action in Altis Life.

Sakzy added a comment.Jan 26 2016, 6:57 PM

V12 crash too.

Error with animation of arma.

I test v13 now.

bump...
any update from BI managers on where this is going?

Same here. Arma 3 Dedicated Server crashes in version 1.54.133741 after 5 - 30 Minutes. Most time when we using a drone or be in a helicopter.

Any news? :-( Hope they will fix it very soon. It is extremely frustrating. We cannot play since some weeks on our dedicated server because the server always crashes during the mission. We have lost some players because of that reason :-(

tested now with Ubuntu 14.04.

No crashes. more then 14 hours stable uptime.

Its a winshit problem. So stop using windows and switch to linux