Playing for period of time leads to low framerate (3 FPS) and never recovers
Reviewed, NormalPublic

Description

After playing multiplayer for a period of time (random), the game will go into extremely low framerate and become unplayable until a restart.

Details

Blocked By
T123227: log 2 with procdump runing
Restricted Maniphest Task
Severity
Crash
Resolution
Open
Reproducibility
Random
Operating System
Windows 10 x64
Category
Multiplayer
Additional Information
focher created this task.Jul 4 2016, 4:59 AM
razazel claimed this task.Jul 12 2016, 9:53 AM
razazel changed the task status from "New" to "Reviewed".
razazel added a subscriber: razazel.

Hello,

are you experiencing same issues on other servers (for example ones that are not Invade and Annex)?

Did you try to verify integrity of your game cache?

I did a verify integrity and it showed no issues. I don't play except on that one server. I think others have reported the same problem so this might be a duplicate to be merged with another bug report.

If it's not much trouble, please try to find out what the situation is when playing on different servers, perhaps the official ones or some other servers, preferably the ones that do not require mods.

Hello again :)

So a plan to deal with this is in the progress, there's something I need you to do so we can progress further.

If you won't mind, please, go through these steps

  • Switch to latest Arma 3 "profiling branch" - Steam Branches
  • Go into "Arma 3" directory in Steam and rename "arma3profiling.exe" to "arma3.exe" (Launcher and BattlEye are expecting that one).
  • Launch game and start playing.
  • Press together {L SHIFT} + {NUM -} (NUM Minus) and release them - Help
  • Then just type "frame" (as with any other cheat mentioned in the Help)
  • Should a dialogue stating it was successful appear, select "COPY ALL" (third button at the bottom row).
  • Data should be copied to clip board so paste them (CTRL+V) anywhere you want (e.g. create a new txt file on desktop and paste the info there)

Once you have that file, please upload it here (or, if you encounter any trouble with that, to some free file sharing service and post the link here).

Thank you very much :)

Also please make sure to perform the cheat "frame" only once the FPS drops and freezing happen.

arma-bug.txt

From a crash today. Interesting is that after I did a COPY ALL, the game engine recovered and I was able to force a respawn to go back into a usable game. This happened without quitting the game and rejoining as I normally have to do.

Another crash report from the FRAME command.

CRASH2.txt

focher added a comment.EditedJul 17 2016, 9:28 AM

These are the system status details from the CRASH2 incident (Sun Jul 17 @ 9:19am). Note the large RAM and pagefile usage. RAM is usually just below 3GB and pagefile less than 5GB.

Lex added a subscriber: Lex.EditedJul 17 2016, 11:54 AM

FPS falls always, when Memory=3000-3500 mb.
When filling of Memory to 3000-3500 mb,
CPU 2-7 always fall to 2-6%
I don't find other reason of this regularity.
Arma3 doesn't work with large volume of Memory, its limit of 3000-3500 mb.
After that, Arma3 can't address to the RAM, there is no record and data reading. The following step - a mistake in any block, and crash of process of arma3.exe.
Time of life of process of arma3.exe depends on what time process will spend on achievement of the size Memory of 3000-3500 mb.
P.S.
Not important how many memory you use in PC.
Any size MaxMEM doesn't influence, process of arma3.exe takes only 3000-3500 mb.
I when used arma3client_performance from Dwarden had the best result. On the stable server it gives +10-20 minutes to process life. The size Memory in process I reached 3450mb.

This is a crash that recovered when I did FRAME.
armacrash3.txt
This is a crash that recovered when I did FRAME.
ARMACRASH4.txt
This is a crash that recovered when I did FRAME.
armacrash5.txt
This is a crash that recovered when I did FRAME.
ARMACRASH6.txt
This is not a crash and was performed IMMEDIATELY after the ARMACRASH6 capture and the game had recovered.
ARMANOCRASH6.txt
After another couple of minutes, my screen went black and I could not use the FRAME capture process. Other players reported that I disconnected with a "Battleye not responding error".

All the captures in the previous comment were during the same game session and on the same server within just a few minutes of each other.

Ronar added a subscriber: Ronar.EditedJul 24 2016, 8:57 PM

Hello, I'm experiencing the same problem as OP. 5fps/LOD after some amount of time, depends on which texture quality I'm using. If ultra - 30 minutes until drop, standard - 1.5h until drop also.

I want to note that I din't have any problems before 1.58 patch and also my framerate was higher than now. But profilling branch fixed that issue going from avg. 20-30 fps to 30-50 fps. Also with profilling branch it feels much smoother (no micro stutters).

Drop occurs on every type of gamemode and different servers: Invade and Annex, Altis Life, King Of The Hill. Profilling branch does not fix this issue, it only improves framerate.

Here is arma crash file : https://www.dropbox.com/s/0nlb0gx3zyy9sgt/ArmaReport_Log_20160724T183420.zip?dl=0
Here is FRAME : arma 3.txt

Frame does not recover my arma to original state.

Edit: Setting textures on whatever settings is not relevant to this case. Found this by setting out ultra and actually I was able to play longer.

Hello,

thank you for the feedback you have provided in this matter.

As progress in this case is moving forward, we are trying to pinpoint the cause, which unfortunately means yet another intel gathering.

A new profiling was published, rev. 137737, that will help us get more precise info about the issue.

Almost all of the performed steps are still the same, almost every of them only two things change:

  • needs to be the latest profiling branch rev. 137737
  • this time include also the rpt file from the session

Once you have those, ZIP them and attach them here again. One frame will do, more of them are very welcome (please make sure to have corresponding rpt as well).

Thank you

Updated to v7 of the profiling branch. Crash was a black screen and 0 fps reading. Memory went to about 8GB steadily (3.2GB used after closing the crashed Arma3.exe). COULD NOT USE THE FRAME CAPTURE PROCEDURE. It did not respond to the steps of LSHIFT+NUM MINUS FRAME.

Arma RPT etc files at https://dl.dropboxusercontent.com/u/17057/Arma%203.zip
Windows crash report at https://dl.dropboxusercontent.com/u/17057/ArmaReport_Log_20160726T204821_User.zip

Two NON CRASH frame captures:

normalstate.txt
normalstate2.txt

after crash CRASH.txt

recovery after crash (about 3-4 minutes waiting and ALT-TABs) CRASHRECOVERY.txt

last crash: ANOTHERCRASH2FPS.txt

RPT: https://dl.dropboxusercontent.com/u/17057/mORE%20Arma%203.zip

Thank you very much :)

focher added a comment.EditedJul 28 2016, 8:40 AM

This is with the v8 profiling exe

Just started the game and joined the Tanoa server. No crash.
nocrash.txt

After playing for a few minutes. No crash.
nocrash2.txt

Crash to 2-3fps.
2fpscrash.txt

Post crash and the engine recovered.
postcrashrecovery.txt

arma rpt file
arma3_2016-07-27_22-00-23.zip

Had the same kind of issue on 2016 July 18 and got an "Out of memory" error while playing a "Liberation" valilla mission on our dedicated clan server [Linux]. Gaming rig "i7-4790/GTX770/16Go @ 1866MHz/SSD Samsung 500".
Believe it was something wrong with my rig before I had seen SITREP report.
As I had clean all files, while looking for an on-board failure, the only hints left are screenshots.
http://imgur.com/a/mLcOB
http://imgur.com/22NGVOR

Ronar added a comment.Jul 29 2016, 3:25 PM

Alright I have bunch of information for you devs. Here are reports. Whole process is done with latest profilling branch version (v10).

After joining Tanoa server : server join tanoa.txt

~15 min after joining : few mins tanoa.txt

1st crash : 1st crash.txt

after crash recovered : 1st recover.txt

Few mins after recover, arma just froze and did not respond. What is interesting game process went to background processes in windows taskmgr. After closing it, arma 3 launcher just gave me crash report. Here it is: https://www.dropbox.com/s/oz14mn9gr2bgt51/ArmaReport_Log_20160729T131300.zip?dl=0

Here is .rpt file of whole session: https://www.dropbox.com/s/yiobklwrjwrtvz1/arma3_2016-07-29_14-02-44.rpt?dl=0

rogerx added a subscriber: rogerx.Jul 30 2016, 2:55 AM
rogerx added a comment.EditedJul 30 2016, 3:06 AM

Ah. I just heard about this and might have part of the golden bee bee.

I usually spend time on the game servers offering the HALO drop scripts. I inadvertently happened to have this bug, not so often as I might add, in the past. One time I happened to be near the flag pole with HALO drop scripting during a significant loss of frame rates, so I clicked on the HALO drop option within the action menu and I escaped this horrid bug. (Being near a HALO scripted point is rare, but map scenario editors/scripters might be able to quickly script something within their action menus to always have a HALO drop option at the ready. Eventually, you'll likely get some good debugging to rule-out or further rule-in possible causes.)

Whatever is causing this bug, the bug can be mitigated by using the HALO drop scripting. Frame rates will return to normal. Now whether the game was stable for hours afterwards, I do not know as this occurred either during RC testing, or just before. If I'm not mistaken, this bug started occurring sometime within the past release prior to 1.62. The bug was so random and so rare initially, and still is rare, it's difficult to wrap one's head around the bug.

I spend quite a bit of time testing open source software, as well as programming for my past 15-20 years, so programmers can rest assured what I've previously stated is a pretty accurate summary and accurate history of this bug. (Unless of course there tends to be more than one of these bugs, being caused by different issues!)

2016.07.29 00:08 UTC: Scanning through the above previous comments, I noticed somebody stating a "COPY ALL" mysteriously alleviated the bug. Could be with all the keyboard activity, we're somehow triggering a copy/paste event within the 3D world, but think this scenario could only occur within the Editor. Shrugs.

focher added a comment.EditedAug 19 2016, 8:54 AM

Played with v15 of the Profiling 1.62 binary. No crash experienced, even after a couple of hours. Here are two sessions of FRAME captures.

Session 1
nocrash3.txt
nocrash.txt
nocrash2.txt
RPT file - arma3_2016-08-18_18-19-45.rpt

Session 2
nocrash1b.txt
nocrash1a.txt
RPT file - arma3_2016-08-18_21-53-45.zip

@focher Could you please try and find also corresponding RPT files?

@focher Could you please try and find also corresponding RPT files?

I added them back into the original comment. Meant to include them the first time!

Lex removed a subscriber: Lex.Sat, Jan 28, 1:45 PM

I have the same issue in the last week. Here is dumpfile, rpt and minidump.

https://1drv.ms/u/s!AqgVT4o86VaQlFnCfA5P6yHT29Vn

rogerx added a comment.EditedMon, Jan 30, 8:59 PM

Switching the memory allocator to (I think) "-malloc=tbb4malloc_bi" if you're on an Intel platform may slightly improve this bug's effect.

I must note, I have not seen this bug present itself upon NAK's Tanoa's server after many hours of game play! As such I have to reasonably speculate this bug is being exhibited by the use of scripts within the many scripts of the map scenario files. However on another very popular Tanoa server, the bug is very persistent and makes itself known quite reliably every hour or so. Knowing how hostile server administrators get sometimes, maybe they're exploiting a known bug, inherently causing this bug.

So best advice or workaround is likely to use another server. I don't know about NAK's Altis server as I haven't been on there for awhile. I've also been using ARMA without any modifications for a couple of months now, and the bug persists with or without modifications.

TOMMEH added a subscriber: TOMMEH.Mon, Feb 6, 5:42 PM
TOMMEH added a comment.EditedMon, Feb 6, 5:45 PM

Video of the bug in action for me here

And I have uploaded my dxdiag here DxDiagTOMMEH.txt

I hope this helps guys!!

Hopefully the following information will help: When I alt tab out and back in, it usually helps to recover normal FPS.

Variable removed a subscriber: Variable.Mon, Feb 6, 5:55 PM

I had the bug happen to me multiple times.
I switched to the Windows allocator "system" via the launcher's Parameter "memory allocator" option.
It never happened in the last months since the switch now.

I run Win10 64bit with an i5, hope it helps a little.

(I used the performance exe the whole time)

This comment was removed by TheBattleLegend.
killswitchh created blocking task Restricted Maniphest Task.Tue, Feb 7, 12:48 AM

dxdiag attached
DxDiag.txt
procdumps 7zipped and attached
Procdump.7z

Alwin changed the status of blocking task T123227: log 2 with procdump runing from "New" to "Reviewed".Tue, Feb 7, 2:25 PM
TOMMEH removed a subscriber: TOMMEH.Tue, Feb 7, 3:49 PM

One important question for you guys.

Did someone who is experiencing this issue got similar issue when playing with disabled BE ?

Lex added a comment.EditedTue, Feb 7, 10:33 PM

Today, at the beginning of the game the FPS was 90, after 2-3 hours was 3 FPS.
ArmaReport_Log_20170207T204627_Lex.zip
@razazel Disconnected battleye, Nobody plays Arma3 without battleye.

ShadowRanger added a subscriber: ShadowRanger.EditedWed, Feb 8, 4:28 AM

I believe this issue may have something to do with the audio side of the game. The reason I say this is because ever since I lowered my ingame audio sources value in the settings to the lowest value, I have not experienced a 4 frame glitch since. Despite this however, I do still get out of memory crashes, but no where near as regularly. Prior to changing this setting I got 4 frames constantly, usually at around 100 minutes of my game running.

Gunzta added a subscriber: Gunzta.EditedWed, Feb 8, 7:45 AM

Here's my experience of this issue. I've never connected to an online server. I always play Arma3 in coop play with small group of friends. I am always the host/server, server is run as internet server and 3-5 friends connect to me. I can nearly always play for about 1-2 hours before I experience the issue. I have i7 6700k OC at 4.6ghz 16GB Ram and GTX 1080. I'm normally running 50-70 fps and then suddenly (perhaps when zooming, like its struggling to read graphics data quick enough) the fps will drop to 3-5 fps. Now here's the thing, thinking this might be a graphics streaming issue, I wait to see if game recovers, and if I wait about 3-5 mins, the game does indeed recover and fps returns to normal, however this is only ever for a limited time and sure enough the problem re-appears in under 30 mins and this time is very unlikely to recover. I have had it recover once or twice after 2nd fps drop but usually by the 2nd time, I'm stuck with 3-5 fps and it never recovers. This has become so I cannot game in Arma3 ever for more than 2 hours without this issue happening. I've had this happen with and without mods but now I know there is a concerted effort to track and crush this bug I will report back with more detailed information the next time it happens.

Lex added a subscriber: Lex.Thu, Feb 9, 12:04 AM

Add Comment