Page MenuHomeFeedback Tracker

Memory leak on every compiled script
Confirmed, NormalPublic

Description

I just learned that all compiled scripts are cached.
Problem is, they are never removed from the game, essentially causing a huge memory leak.
Especially bad for servers that use Extensions to query databases, if they use call compile to cache the result, that means all results stay in memory forever.

Here is a video that shows the effect of adding a onEachFrame handler with 3 lines of code:
https://s.sqf.ovh/2018-12-23_00-24-04.mp4

PS: This is my Christmas gift to you. Merry Christmas.

Details

Severity
Major
Resolution
Open
Reproducibility
Always
Operating System
Windows 8 x64
Category
Scripting
Steps To Reproduce

Execute this in debug console and watch your memory usage

onEachFrame {
_arr = [];
_arr set [500000,random 1215235345];
compile str str _arr;
};
Additional Information

Possible solution would be to remove things from the cache after some time. Or just get rid of the cache. Or somehow make it optional.
Optional would maybe need seperate compile commands though, I don't see that really working out.

Event Timeline

dedmen created this task.Dec 23 2018, 12:08 AM
dedmen updated the task description. (Show Details)Dec 23 2018, 12:24 AM
dedmen updated the task description. (Show Details)

Not really surprising, but passing a string instead of code to an event handler seems to have the same effect, e.g. for a display like in this test case

uiNamespace setVariable ["display", displayNull];
terminateOnEachFrame = false;
onEachFrame
{
	_display = uiNamespace getVariable ["display", displayNull];
	if (isNull _display) then
	{
		if (terminateOnEachFrame) exitWith {onEachFrame {}};
		terminateOnEachFrame = true;
		_parentDisplay = findDisplay 49;
		_parentDisplay createDisplay "RscDisplayEmpty";
		_display = findDisplay -1;
		uiNamespace setVariable ["display", _display];
		_arr = [];
		_arr set [500000,random 1215235345];
		_display displayAddEventHandler ["unload", str str _arr];
	}
	else
	{
		terminateOnEachFrame = false;
		_display closeDisplay 1;
	};
};

@ookexoo Eventhandlers always convert to string and store as string internally. And get compiled before every execution.
https://feedback.bistudio.com/T123355

ookexoo added a comment.EditedDec 23 2018, 11:57 PM

But why don't I observe a memory leak in this case?
Nvm, bad example.

uiNamespace setVariable ["display", displayNull];
terminateOnEachFrame = false;
onEachFrame
{
	_display = uiNamespace getVariable ["display", displayNull];
	if (isNull _display) then
	{
		if (terminateOnEachFrame) exitWith {onEachFrame {}};
		terminateOnEachFrame = true;
		_parentDisplay = findDisplay 49;
		_parentDisplay createDisplay "RscDisplayEmpty";
		_display = findDisplay -1;
		uiNamespace setVariable ["display", _display];
		_arr = [];
		_arr set [500000,random 1215235345];
		_display displayAddEventHandler ["unload", {_arr = []; _arr set [500000,random 1215235345]; str _arr}];
	}
	else
	{
		terminateOnEachFrame = false;
		_display closeDisplay 1;
	};
};
ookexoo added a comment.EditedDec 24 2018, 12:08 AM

Still, the cache is also here an issue. https://feedback.bistudio.com/T123355 would imply that the caching is done independent of whether you pass code or string.

dedmen added a comment.EditedDec 31 2018, 3:53 PM

https://feedback.bistudio.com/T123355 would imply that the caching is done independent of whether you pass code or string.

Nope it doesn't. All eventhandlers are recompiled before execution, no matter what you pass to the addEventhandler command. And they also all get cached.

_display displayAddEventHandler ["unload", {_arr = []; _arr set [500000,random 1215235345]; str _arr}];

You are just creating an array and throwing it away. Never compiling anything.

_display displayAddEventHandler ["unload", str _arr] that's what you wanted to do

Wulf changed the task status from New to Reviewed.Jan 2 2019, 4:30 PM
Wulf added a subscriber: Wulf.

And a happy New Year to you as well.

oukej added a subscriber: oukej.Jan 4 2019, 12:23 AM

The "fix" with the new compile syntax is not really a fix. Noone will update their code to that, same as noone (relatively speaking) updated their 5 year old code to parseSimpleArray.
Also unless that syntax is used in CfgFunctions every Mission launch that has its own functions still clutters the cache.
Not to think about execVM and eventhandlers and addAction and everything else that I can't even think of right now.

All that changes about the memory leak is from "Its Bohemias fault" to "It's mostly Bohemias fault and also partially your fault for not using the right compile syntax"

jeroennot added a subscriber: jeroennot.EditedJan 11 2019, 6:10 PM

Caching also has a security problem as i described in https://feedback.bistudio.com/T133807

Leopard20 added a subscriber: Leopard20.EditedFeb 19 2019, 2:44 PM

Wait, is this why the game's RAM consumption skyrockets on custom missions and mods?!

I hope devs fix it in the next update.

On badly written missions and mods yes.
And yes fix is underway already.

Confirmed fixed in RC 1.90.145.381

Astaroth closed this task as Resolved.Mar 11 2019, 11:45 AM
Astaroth claimed this task.
Astaroth added a subscriber: Astaroth.

Thanks for confirmation. Fixed.

Issue reintroduced in 1.92.145888 perf/prof build. And I also heard that it's on dev branch but didn't have time to check.

Astaroth reopened this task as Confirmed.Jul 17 2019, 11:14 AM

Hi,
thanks for the info.

dedmen added a comment.EditedOct 30 2019, 4:39 PM

It was removed from perf/prof again due to it having the same memory leak issue.

But... Was reintroduced in 1.96 again

https://s.sqf.ovh/2019-10-30_16-36-13.mp4
Go back to main menu, memory usage doesn't go down.
Start a new mission, memory usage doesn't go down. Cache isn't being cleared ever it seems.
Only exiting the game fixes it (obviously)

And fixed in 1.96 hotfix 1.96.146.114
Please don't let it come back until I get my own hands on that ^^