Page MenuHomeFeedback Tracker

Mass Battle Eye Kicks on Multiplayer servers.
Closed, ResolvedPublic

Description

For the past 10 months our 4 servers have had an issue with battle eye kicking everyone off of the server. I was wondering if this was something you are already aware of.

I have done some research into the issue and this is what I have found.

  1. It is not OUR mission file that is at fault. I know this because our servers are not the only affected servers. Popular servers such as Olympus and Asylum are also having these "Mass kicks". I have downloaded about 8 different servers worth of mission files and they are all different. That can almost guarantee that it is not the mission files doing.
  1. Both modded and un-modded servers are having the issue with the battle eye kicks. As I said before I have tried wasteland missions, king of the hill missions, life missions and modded missions. They all have "Mass battle eye kicks"
  1. This is not something that you guys did intentionally, since servers reported having the issues at different times. What I mean by that is Olympus started having mass battle eye kicks in September, Asylum started having kicks in June, A3 wasteland started kicks in October, etc....This concludes that is was not a patch that went out and broke something.

I can tell you that our servers have tried multiple fixes, such as switching databases, buying new hardware for the servers, we even thought 64 bit would fix it. We even re-wrote the mission file in an attempt to see if that would work. All attempts to fix the issues have failed.

I am not sure what exactly is happening since I do not have experience on the network side of things. However, what seems to be happening is that when a large amount of packets are coming through (50 players or more) sometimes the packets do not make it to the client. It ends up blocking up somewhere and then battle eye mass kicks everyone to clear the blockage.

Details

Severity
Crash
Resolution
Open
Reproducibility
Always
Operating System
Windows 10 x64
Operating System Version
Windows 10 pro
Category
Server
Steps To Reproduce

When there is more than 50 players on the server it starts to mass battle eye kick people.

https://gyazo.com/abb5a5d41ed106cb322eb3c1f541e99f

Additional Information
NOTE: I have only noticed the battle eye kicks when there are about 50 players or more on a server.

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

I thought i was the only one having this issue.
Everytime i start my headless client, even if he does nothing, when we reach more than 100 players, after few minutes everyone gets kicked.
That's why i stopped using HC.

Unknown Object (User) added a comment.May 31 2017, 3:31 PM
This comment was removed by Unknown Object (User).
KevinP added a subscriber: KevinP.Jun 5 2017, 10:46 PM
BoGuu added a subscriber: BoGuu.Jun 19 2017, 1:27 AM

any comment from BI on this. seem as normal you guys hide under rocks ?

wallop added a subscriber: wallop.Jun 19 2017, 7:58 PM
KevinP added a comment.EditedJun 20 2017, 5:40 AM

We are experiencing this at around 100+ players, some times it is fine but other times BE randomly starts kicking at around 105~ players...

Unknown Object (User) added a comment.Jun 22 2017, 7:16 PM
This comment was removed by Unknown Object (User).
dedmen added a subscriber: dedmen.Jun 22 2017, 7:58 PM

My server has mass kicks very frequently, so tonight I tried running it without any BE filters. We lasted 4 hrs 15 mins with 70+ players and no issues, then I turned the filters on and within 40 minutes a mass kick occurred.

I'll do more tests tomorrow removing individual filters to see if its a specific type of filter causing my problems and then post my results here.

Unknown Object (User) added a comment.Jun 25 2017, 5:53 PM
This comment was removed by Unknown Object (User).
Norman added a subscriber: Norman.Jun 25 2017, 6:25 PM

We completed testing with no filters, I don't think they are the issue

Do you have any messages in the log before kicks?

Nothing logged other than the log of the kick

Unknown Object (User) added a comment.Jun 28 2017, 4:53 PM
This comment was removed by Unknown Object (User).
Gnashes added a comment.EditedJun 28 2017, 6:35 PM

@Wilco Asylum is keeping an eye on this. Seems based on our internal experiments that a hard restart alleviates the issue, but if left unchecked the server will continue to kick every time it fills back up (every 15 mins or so) until it naturally hard restarts.

This can either mean a single instance if an Admin is on a server when it kicks and hard restarts it, or dozens of times in a few hours if nobody is around and monitoring the servers.

We suspect it has something to do with server memory, and the amount of data being held in memory by Arma for variables, but haven't been able to actually validate that in any substantial way.

MrSmirnoff added a subscriber: MrSmirnoff.EditedJun 28 2017, 6:56 PM

Our life servers are suffering from this as well. Some servers suffer from it more than others even though they all run the same mission. We tried disabling BE filters but we still suffered mass kicks regularly. This is a very serious issues since when mass kicks happen players lose progress and money in game.

Please fix this Bohemia thank you.

Bamf added a subscriber: Bamf.Jun 28 2017, 11:45 PM

Gaming Asylum has extensive logging inside of Splunk of data around the times this issue occurs. I'm happy to provide data to BIS as necessary.

This is one of the largest issues facing the modding community though, and I'm sure anyone with a large server would be more than happy to assist you in tracking this down.

After a few more days of tests I'd like to clear up my last message.
I do still get mass kicks without battleye filters, they are just not as frequent.
As you guys have said its clearly not the cause of the problem, just me shooting myself in the foot with the amount of filters I had.

Azeh added a subscriber: Azeh.Jun 30 2017, 7:48 PM
Erik added a subscriber: Erik.Jul 2 2017, 9:55 AM
TikTak added a subscriber: TikTak.Jul 3 2017, 2:09 PM

This issue is hugely affecting our Arma 3 community and servers. Would love to hear Bohemia's answer on this.

Can you guys give us a hand here? @Alwin @razazel @Jawsh

Azeh added a comment.Aug 7 2017, 1:37 PM

Bohemia pls

Bohemia have commented on this via twitter saying they are still investigating the issue

Unknown Object (User) added a comment.Aug 15 2017, 6:06 PM
This comment was removed by Unknown Object (User).

As mentioned on the Twitter we are looking into the issue together with BE developers, searching for a suitable solution. Unfortunately that is the only info I have for now, I'll try to keep this updated with the progress (once I know something).

Unknown Object (User) added a comment.Aug 29 2017, 8:46 PM
This comment was removed by Unknown Object (User).
Neology added a comment.EditedAug 29 2017, 9:12 PM

We get this issue on a rarer occasion than Wilco above, that said, it's the exact same errors and the same actions taken.

I personally believe it has nothing to do with Battleye too, we see it even more whilst using our headless client, fortunately, we're rewriting its system so get the kick less, however it all points to network activity as shown in the logs above.

I implore @razazel and @Dwarden to take the offers from communities such as Asylum, and get tests done you can't do in-house. We offer our services too!

Unknown Object (User) added a comment.Aug 29 2017, 9:52 PM
This comment was removed by Unknown Object (User).
Bamf added a comment.Aug 29 2017, 10:17 PM

To echo what has been said today, please let me know what sort of reporting you need from us to help you track down this issue. We keeps logs of virtually everything, and have them going back for several years. I am personally happy to help getting this tracked down as it has plagued our community going back to May of last year.

Unknown Object (User) added a comment.Aug 29 2017, 10:53 PM
This comment was removed by Unknown Object (User).
Bamf added a comment.Aug 30 2017, 12:57 AM

@Wilco let me dig through the logs a touch. Looks like it actually began before May. It's been so long since I pinpointed the start date - and I had May in my head for some reason.

Unknown Object (User) added a comment.Aug 31 2017, 7:24 PM
This comment was removed by Unknown Object (User).
Neology added a comment.EditedAug 31 2017, 7:53 PM

Things like this REALLY need to be posted by BIS in the tracker, otherwise, what's the point of it?

Not the first time we get answers through twitter.

Lakotiz added a subscriber: Lakotiz.Sep 3 2017, 9:27 PM
Azeh added a comment.Oct 19 2017, 5:13 PM

maybe one day

l9th9l added a subscriber: l9th9l.Oct 19 2017, 6:49 PM

@Dwarden hey cucklord, any response?

https://dev.arma3.com/post/sitrep-00204

We also have some more information regarding a BattlEye mass-kicking, which happened to some communities over these past few months. It seems we finally found a potential solution to this problem with our senior programming staff working on it as we speak. However, it might still take us some time, because the changes require low-level and potentially risky netcode changes. For this reason the implementation and testing needs to be handled with extreme care. It means we don't have a specific date yet, but we hope to release the change in one of the upcoming Main Branch updates. Please bear with us!

As happy as I am to finally have had something said about it, it took them far too long to even comment on it. Running a server is stressful as it is, chasing the producer of the game certainly doesn't help this.

This really should have been something that was 'being worked on' the day it started happening

Eyad added a subscriber: Eyad.Nov 26 2017, 1:19 PM

unban tyrone

Dell added a subscriber: Dell.EditedNov 26 2017, 3:28 PM

I assume that this is due to poor configuration of basic.cfg.
https://community.bistudio.com/wiki/basic.cfg
Also look at the research on the forum.
Also try to disable BattlEye. I'm sure the problem will remain

Unfortunately not.

See my quote from the SITREP above.

Dell added a comment.Nov 26 2017, 3:34 PM

We faced the same problem. But the correct setting of basic.cfg helped. I do not know why, but I also had off battleye. It seems that because of the dll.

Hey guys, have you by any chance tried the latest RC build? The potencial fix mentioned in the SITREP is already there so, it might be worth a shot giving it a try and reporting back in case something is still broken :)

Azeh added a comment.Nov 27 2017, 12:55 PM
This comment was removed by Azeh.
Neology added a comment.EditedDec 1 2017, 9:23 PM

Still getting the issue:

20:06:33 BEServer: registering a new player #1577864936
20:06:37 NetServer: cannot find channel #1577864936, users.card=46
20:07:06 Server: Network message 115523 is pending
20:07:06 Server: Network message 115523 is pending
20:07:07 NetServer: cannot find channel #516779254, users.card=46

Bamf added a comment.Dec 4 2017, 7:47 PM

The latest update does not appear to have fixed the BE mass kick issue for Asylum. In the last 72 hours we have had approximately 13 such mass kicks with the same messaging as before the 1.78 patch.

Some servers seem to have "found their own fix" you'll have seen them posting here prior but no longer mentioning it (rpuk)

hope you don't mind Bamf, i'm quoting you on our forums as people are saying its "just us"

Actually managed to get a wireshark capture as it crashed, was doing it for my ISP to see if there was anything their end... @Dwarden I can provide if needed.

Azeh added a comment.Dec 5 2017, 12:05 AM

Some servers seem to have "found their own fix" you'll have seen them posting here prior but no longer mentioning it (rpuk)

hope you don't mind Bamf, i'm quoting you on our forums as people are saying its "just us"

Zero-One still have them too.
https://i.gyazo.com/f9c9845ae0e7fea211052b5a2f7a4a1a.png

Thats 5 communities that is suffering with this some have been suffering for over 12 months, can we get some sort of acknowledgement that this issue still exists, you contact bohemia to report it and they say post here, when you do nobody ever responds....

@Neology Haha, feel free. I largely just stopped answering our community and began directing them to this ticket.

RoleplayUK and Grand Theft Altis both have this issue solved. There are many servers out there that are having this issue, Even Bohemia have been looking into it, would be nice if they threw a lifeline. The downside is, these servers are gaining popularity as they haven't got the kicks anymore, why would they want competition?

Azeh added a comment.Dec 5 2017, 6:37 PM
This comment was removed by Azeh.
This comment was removed by Neology.
Neology added a comment.EditedDec 6 2017, 1:48 AM

Seeing as they dont do it themselves:

[7:17 PM] Dwarden: good news about the BE troubles ... and bad news about the BE troubles ...
[7:17 PM] Dwarden: will know more after next DEV branch tomorrow 😁
[7:18 PM] Dwarden: fixed some if not all of it ... lot of brave programmers fallen on the battlefield today

Posted 05/12/2017 (dd/mm/yyyy)

So, GTA never had the issue, they re-coded their mission so much, they cant pinpoint what fixed it because they never had it.

All the Destination unreachables are port 2306
This was captured with Firewall OFFLINE.

An example of log before everyone is kicked:

19:58:27 Server: Network message 5789380 is pending
19:58:27 Server: Network message 5789380 is pending
19:58:27 Server: Network message 5789380 is pending
19:58:27 Server: Network message 5789380 is pending
19:58:27 Server: Network message 5789380 is pending
19:58:27 Server: Network message 5789380 is pending
19:58:28 Server: Network message 57896cf is pending
19:58:28 Server: Network message 57896cf is pending
19:58:28 Server: Network message 5789d5e is pending
19:58:28 Server: Network message 578a84d is pending
19:58:30 Server: Network message 578e1cd is pending
19:58:30 Server: Network message 578e1cd is pending
19:58:30 Server: Network message 578e1cd is pending
19:58:30 Server: Network message 578e1cd is pending
19:58:30 Server: Network message 578e1cd is pending
19:58:30 Server: Network message 578e1cd is pending
19:58:32 NetServer: cannot find channel #1373936035, users.card=72
19:58:32 Message not sent - error 0, message ID = ffffffff, to 1373936035 (ツ| Flames  )
19:58:32 Server: Network message 57909d3 is pending
19:58:32 Server: Network message 57909d3 is pending
19:58:32 Server: Network message 57909d3 is pending
19:58:32 Server: Network message 57909d3 is pending
19:58:32 Server: Network message 57909d3 is pending
19:58:32 Server: Network message 57909d3 is pending
19:58:32 Server: Network message 57909d3 is pending
19:58:32 Server: Network message 57909d3 is pending
19:58:32 Server: Network message 57909d3 is pending
19:58:32 Server: Network message 57909d3 is pending
19:58:32 Server: Network message 5790d50 is pending
19:58:32 Server: Network message 5790d50 is pending
19:58:32 Server: Network message 5790d52 is pending
19:58:32 Server: Network message 5790d52 is pending
19:58:32 Server: Network message 5791043 is pending
19:58:33 Server: Network message 57923c3 is pending
19:58:33 Server: Network message 57923c4 is pending
19:58:33 Server: Network message 57923c4 is pending
19:58:33 Server: Network message 57923c4 is pending
19:58:33 Server: Network message 57923c4 is pending
19:58:33 Server: Network message 57923c4 is pending
19:58:33 Server: Network message 57923c4 is pending
19:58:33 Server: Network message 57923c4 is pending
Azeh added a comment.Dec 14 2017, 3:13 PM
This comment was removed by Azeh.

having the exact same problem. anyone know a fix?

Neology added a comment.EditedMay 26 2019, 11:56 PM

The main issue that caused this was corrected by bohemia months ago... any further issues are usually related to a bad basic.cfg

You need to send more detail on;

  • how many players were online at the time?
  • What type of mission are you running?
  • who's the server host? (e.g hetzner, ovh, etc)
  • server's connection.

and anything else you feel is relevant.

I thought i was the only one having this issue.
Everytime i start my headless client, even if he does nothing, when we reach more than 100 players, after few minutes everyone gets kicked.
That's why i stopped using HC.

3 years later, still the same problem with a different mission.
And still a lot of "Network message 57923c3 is pending" in the logs.
With 100+ players , on a dedicated OVH server
I tried different parameters in basic.cfg but it doesn't seem to make any difference

What type of mission are you running?

ElRabito added a subscriber: ElRabito.EditedAug 27 2020, 8:33 PM

We run a Exile server and have the same problem and i can't find any possible reason for this problem.

We had this problem on 3 different hosters now and all 3 servers had a Gigabit connection. It started happening in April this year before we had no problems.
It starts happening with 45+ players and after arround 2 hours mission runtime.

Hoster: OVH

Current hardware:
-AMD Ryzen 7 3800X - 8c/ 16t - 3.9 GHz/ 4.5 GHz
-64 GB DDR4 ECC 2666 MHz
-2x960 GB SSD NVMe Soft RAID

Most mass kicks now relate to a bad basic.cfg setup.

https://community.bistudio.com/wiki/basic.cfg

Pi added a subscriber: Pi.Jun 8 2021, 5:46 PM

Hi, is this ticket still relevant or can it be closed?

Closed as far as we're aware, @Biground

Pi closed this task as Resolved.Jun 8 2021, 9:49 PM
Pi claimed this task.

Closed as far as we're aware, @Biground

Thanks, I'll close it then
If anyone else feels this issue is still current/active feel free to throw a comment here

This comment was removed by Dwarden.