Anticheat Threshold Tuning as a Repeatable Process

Change one threshold, observe, then decide

Tune thresholds in a fixed loop. Record a baseline, change one value, observe for a set window, label the alerts that came from that window, then keep or revert the change. Changing several thresholds at once makes it impossible to tell which one caused the result. Write each cycle in the change log, so the next decision starts from evidence instead of memory.

Record the baseline before you change anything

A baseline is the alert picture before your change. Count alerts by type and by player for a window that includes your normal peak hours, and note any unusual event in that window. Without this record, a quiet day after a change tells you nothing about whether the change helped.

Run one tuning cycle

Each cycle has six stages. Finish one stage before starting the next, and keep the output of every stage in the change log entry for that cycle.

Worked example: one cycle on a combat check

The baseline for a combat check shows 40 alerts across a weekday evening, with most players receiving one or two. Reviewers suspect alerts cluster after ender pearl use, but nobody has labeled them yet. Stage one labels the sample first. Of 15 pearl-related alerts, 12 are unclear and 3 are real. The team logs the pearl hypothesis, makes one exemption change, and observes the next window with the same reviewers. Only then does the team decide whether the exemption helped.

Decide at the end of each cycle

The end of a cycle is a decision, not a feeling about the numbers. Use the outcomes below and write the one you chose in the log.

• Keep the change when the new window shows fewer alerts and the labels show no new false alerts.
• Revert when alerts fall but real alerts fall too. A lower count that hides real detections is a regression.
• Keep observing for one more cycle only when the stop rules allow it. Otherwise escalate to the rollout owner.

Common mistakes in threshold tuning

Most bad tuning history comes from a few habits. Watch for them in your own log.

• Tuning from a single busy evening and calling the result a trend.
• Changing a threshold and an exemption in the same cycle, which makes the result impossible to attribute.
• Reverting a change without a log entry. An untracked revert looks like a change that was never tested.

Compare windows fairly

Most bad tuning decisions come from comparing windows that are not alike. Fix the comparison rules before the cycle starts.

• Use the same window length before and after the change.
• Compare similar hours. A weekend peak is not a fair baseline for a weekday quiet period.
• Mark scheduled events separately. A large event can raise alert counts for reasons unrelated to the threshold.
• Compare per player as well as per alert type. Ten alerts from one player and one alert each from ten players point to different problems.
• Use the same reviewers, or relabel a sample from the old window with the new reviewers.

Set stop rules before the cycle starts

A common mistake is to keep nudging a threshold because each new value looks slightly better. Fixed stop rules prevent that drift.

• Revert when the change creates new false alerts that reviewers confirm.
• Stop after the agreed number of cycles for one check, then pass the check to the rollout owner.
• Do not change a threshold to address alerts that nobody has labeled yet.
• Do not lower a threshold because one player complained. Review that player's alerts first.

What this process does not prove

The loop shows how alert volume and labels changed on your server. It does not prove that a threshold is accurate in general, that every cheater will be flagged, or that a legitimate player will never be flagged. Keep those claims out of staff notes until you have evidence that goes beyond your own alert samples.