Page MenuHomeFeedback Tracker

my dedicated server is no longer listed in the server list, I see 'No server found'
Assigned, NormalPublic

Description

The same pattern happens all the time since I installed the server last week:

  1. I restart the server
  2. Server is listed in the server list and I join, some other players may join as well
  3. 1 day later or so I try reconnect to server by going to the server list and I can no longer find it with filter and not even with Direct Join by entering the correct IP and port
  4. If I restart the server it is visible in the server list again and I can join it

Configuration:

  • Reforger DS version: 0.9.5.51
  • Command line: -gproj ./addons/data/ArmaReforger.gproj -config /home/steam/mace_reforger_van/reforger.json -profile /home/steam/mace_reforger_van -backendlog -maxFPS 120 -nothrow
  • Server config :
    • ArmaReforgerServer process runs with privileges of normal user 'steam'
    • all files and folders used by the ArmaReforgerServer process are also owned by user 'steam'
    • check attachments reforger.json, mace_reforger_van@.service
    • server started with command: sudo systemctl start mace_reforger_van@alpha
  • HW config
    • Linux OS, HW 16GB RAM CPU Intel(R) Core(TM) i7 920 @ 2.67GHz (configuration: cores=4 enabledcores=4 threads=8)
    • check attachment hw.txt
  • Network config: check attachments listening_ports.txt, firewall.txt
  • Log
    • Mon 2022-05-23 12:27:31 UTC patched dedicated server to 0.9.5.51 and restarted
    • Tue 2022-05-24 17:38:45 UTC server process stopped and I restarted it manually
    • Wed 2022-05-25 20:44:36 UTC dedicated server is not listed in the game server list
    • check attachment logs.txt

Details

Severity
Major
Resolution
Open
Reproducibility
Always
Operating System
Linux
Operating System Version
Linux 4.19.0-20-amd64 #1 SMP Debian 4.19.235-1 (2022-03-17) x86_64 GNU/Linux
Category
Dedicated Server
Steps To Reproduce
  1. I restart the server, it is visible in the server list again and I can join it
  2. Server is listed in the server list and I join, some other players may join as well
  3. 1 day later or so I try reconnect to server by going to the server list and I can no longer find it with filter and not even with Direct Join by entering the correct IP and port
  4. If I restart the server it is visible in the server list again and I can join it

Event Timeline

d3nn16 created this task.May 25 2022, 11:52 PM

Geez changed the task status from New to Assigned.May 26 2022, 12:07 PM
d3nn16 changed Category from General to Dedicated Server.May 29 2022, 12:04 AM

I noticed for the second time that, as time passes by (after one or two days), the reforger dedicated server process ends up using almost all memory (94%, check attachment) and ssh session becomes slow and later even impossible (ssh requests timeout). I have to restart server from the web management console.

See https://community.bistudio.com/wiki/Arma_Reforger:Server_Hosting :

As of 0.9.5 it is heavily recommended to use this startup parameter, set to a value in the 60..120 range; otherwise, the server can try to use all the available resources!
ArmaReforgerServer.exe -maxFPS 60

Have you tried lower it to 60?

d3nn16 added a comment.EditedMay 31 2022, 2:15 PM

I changed maxFPS from 120 to 60 and I notice the same behaviour : over less than 24 hours period the memory usage increases (eg from 5 GB to 12 GB as I write) although only one player connected to the game at the beginning of this period and no other players connected afterwards.

It looks to me as if the server process has a memory leak.

Should I fiddle around with a memory allocation parameter? What would be the recommended setup for Linux server?

UPDATE May 31st
In the end, setting maxFPS 60 doesn't change anything to the outcome (15GB memory used, logs indicating failed heartbeats and "Server not found" message when trying to join my dedicated server from the game)

barf added a subscriber: barf.Jun 1 2022, 7:17 PM

Duplicate of T165158 ?
Remember to search for existing bugs before filing new ones!

d3nn16 added a comment.Jun 1 2022, 9:52 PM

Not duplicate, just read the title of this ticket. And yeah I searched before for existing tickets and didn't find yours because at that moment I wasn't aware of this memory leak problem (I discovered it afterwards).

Your ticket T165158 is a possible cause for my issue (memory leak => 'No server found' message)

So I linked this issue with yours as a 'subtask' which I understand is a possible 'cause task'.

d3nn16 added a comment.Jun 2 2022, 6:46 PM

Yesterday I patched the server to 0.9.5.61 version, emptied the .save folder, changed the mission to {1AEE808E7689FE54}Missions/23_Campaign_NWCoast.conf, restarted the server and launched the mission by connecting to the server then disconnected.

At the beginning the server CPU usage was pretty high compared to before the patching (>100% compared to <10%).
9 hours later error messages appear in the logs (again related to heartbeats but a bit different).
At the same time as the log error messages the CPU usage dropped to <1%, memory leaking stopped at 8GB and the server is not listed in the servers list anymore.

I noticed 2 new parameters in the server JSON config file and added them to server config, then restarted.
"a2sQueryEnabled": true,
"steamQueryPort": 17777

d3nn16 added a comment.Jun 6 2022, 1:09 AM

Some players (not more than 5) joined the server today and played about 1 hour before all of them lost connection with server at the same time.

After that point :

  • the server was no longer listed in the server list
  • I lost remote ssh connection to the server
  • the Reforger server process was using only 9GB of RAM (well below the 16 GB limit)
  • some errors related to Http request appeared in the logs (see attached logs since server last restart)

Sun 2022-06-05 22:26:38 UTC mace_reforger_van_alpha[20342]: BACKEND (E): Http Error (SEND)
Sun 2022-06-05 22:26:38 UTC mace_reforger_van_alpha[20342]: BACKEND (E): [RestApi] ID:[438] Error - Cannot send request
...
Sun 2022-06-05 22:27:23 UTC mace_reforger_van_alpha[20342]: BACKEND (E): [RestApi] ID:[439] Error - Cannot send request
Sun 2022-06-05 22:27:23 UTC mace_reforger_van_alpha[20342]: BACKEND (E): Http Error x-request-uid="1e6b0280-f3c1-4e35-9f34-881608b31076", apiCode="MpRoomNotFound", message="There is no room with id ar-steam"

The server was configurer with the property "dedicatedServerId": "ar-%profilename" which seems to generate "ar-steam" value found in the logs. So I set the property to empty string and restarted the server.

Any clue of what is going on?

Uro added a subscriber: Uro.Jun 7 2022, 3:51 PM

Please do not parent unrelated issues to your own to try and raise the profile of your issues, it is disingenuous behaviour at best and entirely misleading for anyone following tracker issues!

My issue was a completely seperate issue reported to a developer on Discord who asked me to provide a feedback tracker ticket for it.

Uro removed a subscriber: Uro.Jun 7 2022, 3:55 PM
d3nn16 added a subscriber: Uro.Jun 8 2022, 12:16 AM

@Uro
You accuse me of trying to raise my ticket profile by linking unrelated issues which I deny
I related your issue since it looked like the root cause of the problem I am facing (memory leak => server not found in the server list)
Btw it is someone else who send me the link to your post because he also thought it was related to my issue.
If you don' want to link your issue to mine, fine with me but that won't help solve our issues which do seem related

Please next time refrain from accusations

Uro removed a subscriber: Uro.Jun 15 2022, 1:43 PM