there is a similar topic about this here:
http://feedback.arma3.com/view.php?id=21550
- Queries
- Arma 3 Activity
- All Stories
- Search
- Advanced Search
Advanced Search
May 10 2016
ECID_1_30_stable_in_arma3server_exe
crashed after 9 hours. Had approx. 60 people running through 7 different missions loaded via rotation.
ECID_1_30_stable_in_PhysX3_x86_dll_1
60 players max ran through 5 different missions. Did crash after 18 hours. Last 6 hours the server did not have any players on it.
ECID_1_30_stable_in_PhysX3_x86_dll_2
Ran for 18 hours. Had 60 players on it. Last 8 hours there were no clients connected on it.
That is supposed to state "instances of crashs". Should have been more thoughtful with wording - fixed in above thread.
We are running a maximum of 3 production arma3Servers per physical server. For a grant total of 5 production arma3 servers and 2 testing instances (only active when needed) .
Even if it where "8 instances on 5 servers", that is 1,6 arma instances per machine. But lets not split hairs here.
This is not a "hardware overloaded issue". It is about Arma3Server gobbling Ram like its candy, 6 times faster then it used to do (1.24), and the server then becoming stuck and not even crashing, so you can only see that something is wrong, by you joining the server and getting infinite loading screen.
pretty sure this has been fixed since 1.30 . At least it has not appeared since then.
running one server without BE now. Will report back with news.
The BattlEye disabled server has now become stuck in the same manner.
disabling BattlEye did, as predicted, have no bearing on the servers crashing.
Still happening. we have got 8 occurances of crashs on 5 different servers (windows 2012 / Ubuntu Wine) where it has happened during last 2 days.
I managed to create a Manual Dump of a stuck arma3server.exe:
Taskmanager --> Detail --> Arma3server.exe -> Create Dump File.
It is 90Mb packed in RAR (1.7 GB unpacked). It is available on this download link:
https://www.dropbox.com/s/dciovms1qu5vwph/arma3server.manual.Dump.rar?dl=0
When i then tried to analyze the "wait chain", it caused the arma3server.exe to crash and create a crashdump.
It is available here:
https://www.dropbox.com/s/6zuvik7y91dsixk/server_4_Crash_-08_09_2014_infinite.zip?dl=0
this issue does not occur anymore since 1.30 hotfix.
network pending. Yes, bigtime. every 5-6 hours on avarage (depending on how busy the servers get.
https://www.dropbox.com/s/c6y24evkhwkim18/128Prof1__17_40__desynch_people_crashing.zip?dl=0
@17:40 you can see another network message pending issue, followed by 46 people being force disconnected.
Another unknown module crash:
https://www.dropbox.com/s/xa8a7em0ji5da5u/server_4_unkown_module.zip?dl=0
Prof2 RPT with network logging on - 10 seconds. corresponds to the last 2 ASM pictures.
includes 2 instances of Desynchs
The file is available for download here (75 MB zipped)
https://www.dropbox.com/s/tfmo54ch5vqi6cq/arma3server_prof2.zip?dl=0
Servers were updated as suggested to the following versions:
Server Version is latest SteamCMD (1.28.0.127008)
Client Version was latest Steamupdate (1.28.0.127008)
We did encounter another massive choke, that felt "like someone pushed a pause button".
As i was not present immediately as it happened, and the server did not crash, it seemed to have recovered after 5-7 minutes.
RpT's again show a massive amount of log spam like the examples below:
2014/09/04, 19:50:38 Server: Network message 805114c is pending
2014/09/04, 19:50:38 Server: Network message 805114c is pending
2014/09/04, 19:50:38 Server: Network message 805114c is pending
The RPT was uploaded on this feedback tracker ticket under file name
zip file icon arma3server_2014-09-04_03-00-00.zip [^] (379,994 bytes) 2014-09-04 20:28
Same server runtime as last report.
Network message cc94e6 is pending Log-spam and outgoing network bandwidth spike. Server felt like it was trying to catch up on a hughe amount of simulation. And slowly recovered. As the Bandwith consumption dropped, so did the "movement" of players on the Ingame map resume.
The server eventually crashed:
Fault address: 114C68B0 00:114C68B0 Unknown module
Crash dump has been added as
server4_unknown_module_Crash.zip
Uploaded a graph of what the hardware was doing when this issue occured.
CPU usage increased by 30%, Network usage decreased by 0.6 MB/s, Disk IO increased by 6 Million IO's
Keep in mind it is a E3 1245v2. running windows 2012, and nothing but 1 Arma3-server. Times coincide with the RPT logspam of pending network messages.
The reason the server probably did not crash this time, is because i am able to handle a lot of io's, as long as the written data does not surpass 2048 MB of Data.. (Primocache server edition. 2GB Assigned Ram cache layer between SSD and Operating system.
Confirmed. Issue no longer resent.
04/09/2014 19:47-19:51
We started experiencing Client CTD's again. ON 3 Servers a combined total of 83 people got kicked.
Server Version is latest SteamCMD (1.28.0.127008)
Client Version was latest Steamupdate (1.28.0.127008)
There were at least 4 more admins experiencing the same issue, at the same time, on the skype admin channel group.
We just lost concurrently 85 players on 2 seperate Arma3 servers on 2 seperate dedicated boxes. RPT's all show the beclient.dll as source
Added new crashdumps.
This occurred on the latest Stable client binary, For me. Server is restricted to clients >= RC3 (Buildnumber 128958)
Let me reitterate, the 5-10% CTD rate was a false positive.
In the end, it affected 17 out of 21 people.
It emerged avalanche style.
In the meantime Servers have been restarted a couple of times:
25/08/14 17:41 Server <5> Crash
https://www.dropbox.com/s/srl3x3coboh765f/Crash_5_25_08_2014_17_41.rar
It created Crashdumps with a filesize this time.
When i noticed @ 18:21, i clicked okay on the Application Crash notification. It immediately crashed Server <4> (same machine)
25/08/14 18_21 Server <4> Crash
https://www.dropbox.com/s/1lv2p1gtyopsw8l/Crash_4_25_08_2014_18_21.rar
It created Crashdumps with a filesize this time.
The RPT's look like they are related.
I've uploaded 4 more Crashdumps. "Crash_4_27_08_2014___4StackOverflowCrashDumps.rar"
The Frequency of the stackoverflow related crashes is increasing rapidly.
I have a hunch this might be related to the proliferation of Client_RC, Client_Perf1 and client_Perf2 users trying to join the server.
When Players on those client-versions get killed on the server (1.26 vanilla), they encounter a CTD.
The last 2 crashes documented in crashfile "Crash_4_27_08_2014___4StackOverflowCrashDumps.rar", conincided with a player getting killed and receiving a CTD, they both were client_Perf1 and Client_Perf2 respectively.
The RC2 that Dwarden has has published, fixed the Issue. No more Stackoverflow's causing Server-Crashes.
Thanks.
----
Steam-beta-key: Hotfix126Arma3
[27.08.2014 18:22:28] -[EUTW]- E.C.I.D.: so, how do i get the proper server binaries ?
[27.08.2014 18:23:00] Dwarden (David Foltyn): https://www.dropbox.com/sh/582opsto4mmr [^] ... ixRC2?dl=0
requiredBuild = 126951; Require clients joining to have at least build 126951 (1.26 Hotfix2) of game, preventing obsolete clients to connect
guaranteedUpdates = false; // Experimental Command by Dwarden
Î checked the Ôther 4 servers for the RPT's and Bidumps, and they all are 0 Byte sized.
I am assuming, this is a "Feature" of this Crash ??
A crash like this has not occurred naturally since this report.
I have not found a way to reproduce it as of yet. The only thing that they all (the 5 servers that crashed) have in common, is the Rcon-Log entry indicating the server trying to contact a BE-master and failing in doing so.
just uploaded a armaa3s4.zip
includes:
-keys
-mpmissions (3 sample missions of the variant we are playing)
-basic.cfg
-server.cfg
-StartParameters.cfg
ps: this complete package was linked under "additional information" when the issue was created.
RC3 now causes client CTDs:
http://feedback.arma3.com/view.php?id=20488
Server binary is stable.
No desyncs noticed (we got to 37 players at peak)
Dwarden just posted server-binaries to RC3. Will upload them and let you know how it performs
[18:09:54] Dwarden (David Foltyn): hotfix RC3, 126958 ready
https://www.dropbox.com/sh/582opsto4mmr8d8/AADJZAW3d03pfKpyKJdVNGRca/126hotfixRC3?dl=0
1.28.126.951 --> client
1.28.0.126951 --> Server
It was published yesterday
27.08.2014 18:23:00 by Dwarden on Skype-dedicated server group.
Steam just downloaded a new RC. 16.2 MB big.
1.28.0.126958 --> client
This update crashes the client. What happens is this:
You select server from server-browser. You join server. You select faction. You select your class. You click okay. Instead of loading into the server you get a CTD.
We are running RC2 now ('Hotfix126Arma3'). No more issues with Desyncs.
------------------
Steam-beta-key: Hotfix126Arma3
[27.08.2014 18:22:28] -[EUTW]- E.C.I.D.: so, how do i get the proper server binaries ?
[27.08.2014 18:23:00] Dwarden (David Foltyn): https://www.dropbox.com/sh/582opsto4mmr ... ixRC2?dl=0
requiredBuild = 126951; Require clients joining to have at least build 126951 (1.26 Hotfix2) of game, preventing obsolete clients to connect
guaranteedUpdates = false; // Experimental Command by Dwarden
to give you some feedback:
guaranteedUpdates = false; |
Fixed the 10000 Desynch bug.
The issues described are still present.
We tested the Perf1 Client + Server on Dwardens Chimera H Server with the above referenced game-mode. We could not get enough people together to test it properly (we imho had 8 at maximum).
> its because of the fact that people need to download a client_binary and it requires effort.
values[] does. via Mission Rotation. It reads correctly values of 0.1, 0.2, 0.5, 0.7, etc.
check the last part of the repro steps i posted. (last 25 lines)
What i am saying is, that if values[] supports Int + float, so should default.
Or if Integers only is preferred, like it set in default, then values[] should only allow int as well.
Or else someone scripting (like me) will have to spent hours to figure why a default float value is not read properly, instead is replaced with 1 or 0 depending on the round();
The mission we were playing is inside the Arma 3.rar - including the Crashfiles.
The installscript.vdf is the file that was updated by steam on the server with the game verification.
The error seems to be fixed for most.
Solution was to verify the gamecache on the server.
still wondering what changed that file tho. And how come it only affected a few players, not all of them.
After most people have left the server, i ran a Steam gamecache verification on the Server. It updated a single file: installscript.vdf (only one with the date changed)
Now those people - me included can join the server again. (tested with 7 different players that had the same issue)
related to : http://feedback.arma3.com/view.php?id=19177
Reproduction steps:
Join a server.
Go into the middle of nowhere (out of direct range)
Populate one side with >= 20 players.(the more players receiving the worse it gets)
Speak on sidechat/global/command (broadcast channels)
VON is received choppy.
Server side Settings of Von Quality in steps of 1-0, 11-20 and 21-30 seem to have no effect on choppy/stuttering VON.
can confirm this as still happening.
You buy white smoke grenade you get 2 of em.
You clear inventory
Yo buy 1 white smoke and 1 red smoke:
you get 2 white and 2 reds
you buy 1 white smoke, a red smoke and a IR grenade, a chemlight, one of each magazine.
You get 2 white smokes, 2 red smokes, 2 chemlight, 1 IR grenade, 1 of each magazines.
Could we get this fixed pls ?