Visitar URL original
fpsdet/docs/benchmark.md at main · Nimdy/fpsdet · GitHub
Skip to content

Latest commit

 

History

History
393 lines (315 loc) · 52.5 KB

benchmark.md

File metadata and controls

393 lines (315 loc) · 52.5 KB

FPSDET Benchmark v1

Generated by fpsdet benchmark report from the committed artifacts (fpsdet.benchmark/1; release manifest benchmark/release.json, fpsdet.benchmark-release/1). Do not edit by hand: change the artifacts, then regenerate.

FPSDET Benchmark v1 is a version of the benchmark: its datasets, selections, draws and expectations. It is not a version of fpsdet, whose own version (0.3.0 at this release) changes independently; a later fpsdet is checked against the same benchmark and reported as MATCH, PROVENANCE_ONLY or DRIFT.

What this benchmark can prove

  • That the committed numbers follow from the committed artifacts: every result, evaluation or per-draw statistics file, strength estimate, split and spread verifies by digest, and its numbers recompute, without re-scoring anything.
  • That fpsdet's decisions on the TF2 and CS2 datasets follow from the prepared inputs: the same events, labels, cohort and profile give the same cases, packets, eligibility, evaluation and strength, on Python 3.11 and 3.12.
  • That the prepared inputs follow from the frozen sources by public rules, with no private key: anyone holding the same downloads rebuilds tf2-rgl-v2's data and all nine baseline draws, and cs2cd-v2's data from its one pinned revision, byte for byte. A download that differs from the frozen source is reported as SOURCE_CHANGED, part by part, never used as if it were the same.
  • Exactly what changed when anything moves: the upstream source (SOURCE_CHANGED, with the digest pinned and found), the selection, the code, the ids, the inputs, labels, profile or cohort, the detector code, decisions, findings, eligibility, statistics, strength, or only presentation.
  • How much a TF2 number depends on which never-banned players build the baseline: nine draws fixed before any was run, all reported, none preferred.
  • That every native detector trips on a behaviour planted for it and stays quiet on an honest twin, and that the challenge and external-signature protocols behave as documented, in controlled synthetic fixtures.

What this benchmark cannot prove

  • How well fpsdet detects cheating in general. Each real dataset is one population with someone else's labels: RGL's bans and the CS2CD authors' judgement, neither of them ground truth.
  • Anything about a detector a dataset cannot observe. On TF2 and CS2 most detectors are not observable at all; that is a gap in the data, not a zero.
  • That a TF2 number is a fixed property of fpsdet. It moves with the baseline draw; the benchmark reports the median and the range over nine draws, and nine draws show the spread, not its limits.
  • That upstream will keep serving the frozen sources. logs.tf and RGL can edit or remove what was fetched; the digests detect it (SOURCE_CHANGED) but cannot restore it, and nothing downloaded is committed.
  • That synthetic fixtures resemble real cheats. They are planted by construction and qualify code, never a rate.
  • That fpsdet is better or worse than any other system. Nothing else is evaluated here.

There is no overall score. Each section answers its own question, and a detector a dataset cannot observe is shown as not observable, never as zero.

Every statement below carries its class, and no sentence mixes two:

  • [controlled fixture] planted by construction in synthetic fixtures and scenarios; it qualifies code, never a rate.
  • [benchmark v1 real data] measured on a current dataset of this release, rebuildable from public sources without any private key.
  • [historical real data] a run published before this release, still verifiable, superseded, and never the benchmark result.
  • [architecture limit] what no fpsdet check can see, by design or by a controlled scenario that gets through.
  • [unmeasured] what the benchmark's data cannot show at all, which is not the same as finding nothing.

1. Benchmark datasets

Dataset Role Class Question Result Status against the pin
synthetic-v1 this release synthetic Does each detector trip on the behaviour planted for it and stay quiet on honest twins, and do challenges and external signatures behave as documented? sha256:98b0d3bf54585f9a… MATCH
tf2-rgl-v2 this release real Among RGL accounts fpsdet could compare, how often did each detector fire on cheating-ban labelled accounts against never-banned ones, and how much does that depend on which never-banned players build the baseline? sha256:485d2720a2dc1be3… (spread of 9 draws) MATCH
cs2cd-v2 this release real Among CS2 players fpsdet could compare in one match, how often did each detector fire on hand-labelled cheaters against hand-reviewed players? sha256:e840edb0b7c96b3f… MATCH
tf2-rgl-v1 historical, superseded by tf2-rgl-v2 real Among RGL accounts fpsdet could compare, how often did each detector fire on cheating-ban labelled accounts against never-banned ones? sha256:bb41b1404bd4198d… MATCH
cs2cd-v1 historical, superseded by cs2cd-v2 real Among CS2 players fpsdet could compare in one match, how often did each detector fire on hand-labelled cheaters against hand-reviewed players? sha256:88263520c366c1cc… MATCH

TF2 league matches with RGL ban labels (tf2-rgl-v2).

  • Source: logs.tf: match logs that TF2 community servers upload: each player's shots and hits per weapon, sniper headshots, and kills per minute; RGL: the competitive league's public ban list, with a reason for each ban. Fetched 2026-10-04, frozen at 2026-10-03. RGL's ban list grows and its reasons can be edited, and a player's logs.tf match list grows with every match. The selection reads only bans dated on or before the freeze and matches played on or before it, so a later download selects the same accounts and matches, unless upstream changed what it already held: an edited or removed ban, or a removed or re-uploaded log. prepare reports that as SOURCE_CHANGED, with the digest it expected and the one it found, and never builds on it silently.
  • Licence and use: Neither site publishes terms for its data. logs.tf's robots.txt asks crawlers to stay off its API, so the fetcher asks for a bounded sample, one or two requests at a time with a pause after each, names this project in its User-Agent, and caches every response. Nothing downloaded is committed, and nothing per player: this dataset's ids are public hashes of SteamIDs, so only aggregate numbers and digests are published. No log, ban, SteamID, name, match id, per-player row, case or split list. Data: logs.tf match logs and RGL's public bans. Not affiliated with Valve, logs.tf or RGL.
  • Archive: manifest and digests only: neither logs.tf nor RGL grants redistribution, so a rebuild must fetch from upstream. The digests say exactly whether it got the same data.
  • Ids: None, on purpose. Each SteamID3 becomes tfb- and the first 12 hex digits of SHA-256("fpsdet.tf2-benchmark-identity/1", NUL, SteamID3). Anyone can compute it, so a third party's rebuild has the same ids and the same baseline draws without any key. Because anyone holding a SteamID can also find its id, nothing keyed by it is published. A benchmark identity is not pseudonymization. A deployment that stores or shares player ids should use a keyed hash under a secret key, as tf2logs.py does, never this public one.
  • Selection, frozen (fpsdet.benchmark-selection/tf2-rgl/2): Freeze: RGL bans dated after 2026-10-03 are not bans here, and no match played after it is selected. Labelled accounts: accounts RGL banned for cheating (not for assisting, distributing or developing cheats) with at least 20 team matches on logs.tf before the ban; their 20 latest team matches before the ban. Extra never-banned players: everyone seen at least 3 times in those matches whom RGL had not banned, taken in the order of SHA-256("fpsdet.tf2-honest-order/1", NUL, benchmark identity) until 300 have at least 20 team matches up to the freeze; their 20 latest. Comparison: every never-banned player with at least 8 matches among the selected ones, which includes the 300 extra players. Equal evidence: every player keeps at most their 20 latest selected matches; a banned account's only before its ban. A team match has 12 or more players. Only aimed, hitscan weapons are converted. No private key is read anywhere: the ids, the extra players and every baseline draw follow from public hashes.
  • Counts: bans 3,457, labelled accounts 228, extra never banned 300, selected logs 10,149.

CS2 matchmaking (CS2CD) with the dataset authors' labels (cs2cd-v2).

  • Source: CS2CD on Hugging Face: 795 CS2 matchmaking matches parsed from server-recorded demos, with the authors' cheater labels. Fetched 2026-10-06, revision 44e5129654508b22802a050a45bcdbb44b103d87. Pinned: every fetch reads revision 44e5129654508b22802a050a45bcdbb44b103d87 (2025-05-19) and nothing else; if the hub no longer has it, the fetch stops with SOURCE_CHANGED instead of taking the latest. Each event file is checked byte for byte against its git blob id, and each tick table's size and LFS SHA-256 as the hub reports them at that revision (byte for byte with cs2fetch.py --full). The cached tick tables are rewritten by pyarrow, so only the event files, which the fetcher writes deterministically, are part of the prepared source identity.
  • Licence and use: CS2CD is published under CC BY 4.0. Nothing downloaded is committed. Per-player evaluation rows with the converter's match-local ids, and aggregate numbers, are. Data: CS2CD by its authors, CC BY 4.0. Not affiliated with Valve or with the dataset's authors.
  • Archive: raw source legally redistributable: CC BY 4.0 allows copies with attribution. None is kept here; the pinned revision on the hub is the archive, and the source list's digests say whether a fetch got it.
  • Ids: CS2CD names players Player_N within each match; the converter writes them as -p (nc047-p9). There is no key, so a rebuild gets the same ids. A player cannot be followed across matches.
  • Selection, frozen (fpsdet.benchmark-selection/cs2cd/2): Revision: 44e5129654508b22802a050a45bcdbb44b103d87 of CS2CD, and no other. Matches: the first 180 no-cheater and the first 120 with-cheater matches by number, keeping complete ones (13 rounds or more). Baseline: the first 120 complete no-cheater matches, screened for whole lobbies of outliers by fpsdet baseline --screen-matches. Their players are never scored. Scored: the other 45 complete no-cheater matches (unreviewed) and the 108 complete with-cheater matches (hand-labelled cheaters and hand-reviewed players). Every player is one match.
  • Counts: matches fetched 300, complete matches 273, events 4,071,829, scored players 1,529, baseline players 887; labelled cheater 504, clean, reviewed match 576, clean, unreviewed match 1,649.

2. Label semantics

TF2 league matches with RGL ban labels (tf2-rgl-v2).

Label Role Who What it does not mean
RGL cheating-ban labelled positive RGL banned the account on or before 2026-10-03, and the reason mentions cheating but not assisting, distributing or developing cheats. Each had at least 20 team matches on logs.tf before the ban, and its 20 latest were fetched. That the player cheated in every scored match, or in any particular one. A ban is a league decision, sometimes mirroring another league's.
never-banned comparison comparison RGL had not banned the account by 2026-10-03. It played in the same lobbies as labelled accounts, with at least 8 matches in the selected logs; 300 of them, taken in a public hash order, also have their own 20 latest team matches. In each baseline draw, a public hash sends about half of them to build the baseline instead of being scored. That the player never cheated. A few unlabelled cheaters are likely among them.
banned for something else other RGL banned the account on or before 2026-10-03 for something other than cheating, such as an alt account. Seen in the selected lobbies, with no minimum number of matches. Anything about cheating.
VAC ban, mirrored by RGL other RGL mirrored a Valve anti-cheat ban, possibly from another game, on or before 2026-10-03. Seen in the selected lobbies, with no minimum number of matches. That the player cheated in TF2.

CS2 matchmaking (CS2CD) with the dataset authors' labels (cs2cd-v2).

Label Role Who What it does not mean
hand-labelled cheater positive A player the dataset authors labelled as cheating, after reviewing a match that contained a VAC-banned player. A ban. The labels are the authors' judgement.
hand-reviewed, not labelled comparison A player in a reviewed match whom the authors did not label as cheating. They shared a lobby with a cheater. Proven clean. It is a reviewer's call, but a direct one.
unreviewed match comparison A player in a match with no VAC-banned player. Not reviewed: the authors' spot check of 50 such matches found 97.2% of players showing no cheating. That anyone looked at this player. A few cheaters are likely here.

3. Controlled detector qualification

[controlled fixture] Controlled synthetic qualification: planted by construction, never a real-world rate. Each detector must trip on its planted player and stay quiet on every honest twin; the states are every eligibility status its fixtures show.

Detector Planted (fires) Honest twins (quiet) Eligibility states shown
speed demo:weight-cheat (yes) demo:legal-heavy, demo:blasted, demo:glitch, demo:adrenaline (yes) disabled, eligible (fired), eligible (quiet), insufficient_samples, telemetry_unavailable
fire_rate demo:fire-rate (yes) fixtures:fx-trigger-twin, fixtures:fx-held-trigger (yes) disabled, eligible (fired), eligible (quiet), insufficient_samples, telemetry_unavailable
metronome demo:metronome (yes) fixtures:fx-steady-twin, fixtures:fx-held-trigger (yes) disabled, eligible (fired), eligible (quiet), insufficient_samples, not_applicable, telemetry_unavailable
recoil_floor demo:no-recoil (yes) demo:modded-recoil, demo:late-compensate (yes) disabled, eligible (fired), eligible (quiet), insufficient_samples, telemetry_unavailable
mirror demo:mirror-script, demo:clone-source (yes) demo:late-compensate (yes) conflict, eligible (fired), eligible (quiet), insufficient_samples, telemetry_unavailable
recoil_learned fixtures:fx-recoil (yes) fixtures:fx-recoil-twin (yes) baseline_too_thin, eligible (fired), eligible (quiet), insufficient_samples, not_applicable, telemetry_unavailable
accuracy demo:rage (yes) demo:elite-human, demo:weak-human, demo:reported-streamer (yes) baseline_too_thin, eligible (fired), eligible (quiet), insufficient_samples, telemetry_unavailable
headshot_rate demo:rage (yes) demo:elite-human, demo:weak-human, demo:reported-streamer (yes) baseline_too_thin, eligible (fired), eligible (quiet), insufficient_samples, telemetry_unavailable
median_distance demo:rage (yes) demo:elite-human, demo:weak-human, demo:reported-streamer (yes) baseline_too_thin, eligible (fired), eligible (quiet), insufficient_samples, telemetry_unavailable
geometry_rate fixtures:fx-geometry (yes) fixtures:fx-geometry-twin (yes) baseline_too_thin, eligible (fired), eligible (quiet), insufficient_samples, telemetry_unavailable
extra fixtures:fx-kills (yes) fixtures:fx-kills-twin (yes) baseline_too_thin, disabled, eligible (fired), eligible (quiet), insufficient_samples, telemetry_unavailable
rank_tail demo:rank-outlier (yes) demo:elite-human, demo:weak-human, demo:new-gun (yes) baseline_too_thin, eligible (fired), eligible (quiet), insufficient_samples, not_applicable, telemetry_unavailable
view_snaps fixtures:fx-snaps (yes) fixtures:fx-snaps-twin (yes) baseline_too_thin, eligible (fired), eligible (quiet), insufficient_samples, telemetry_unavailable
acquire_timing fixtures:fx-acquire (yes) fixtures:fx-acquire-twin (yes) baseline_too_thin, eligible (fired), eligible (quiet), insufficient_samples, telemetry_unavailable
supporting_extra fixtures:fx-reaction (yes) fixtures:fx-reaction-twin (yes) baseline_too_thin, eligible (fired), eligible (quiet), insufficient_samples, telemetry_unavailable
account_jump demo:account-changed (yes) fixtures:fx-improved (yes) eligible (fired), eligible (quiet), insufficient_samples, telemetry_unavailable
hidden demo:wall-eye (yes) demo:angle-holder, demo:real-fight, demo:listened (yes) conflict, eligible (fired), eligible (quiet), insufficient_samples, telemetry_unavailable
quiet_aim demo:quiet-radar (yes) demo:steady-hands, demo:listened (yes) conflict, eligible (fired), eligible (quiet), insufficient_samples, not_applicable, telemetry_unavailable
wire demo:wire-lock (yes) demo:picture-track (yes) eligible (fired), eligible (quiet), insufficient_samples, telemetry_unavailable
occluded_motion_replay demo:replay-lock (yes) demo:real-fight (yes) eligible (fired), eligible (quiet), insufficient_samples, telemetry_unavailable
leftover demo:clone-source, demo:clone-buyer (yes) demo:late-compensate (yes) baseline_too_thin, eligible (fired), eligible (quiet), insufficient_samples, telemetry_unavailable
voice demo:radar-friend (yes) demo:callout-friend (yes) eligible (fired), eligible (quiet), insufficient_samples, not_applicable, telemetry_unavailable

22 of 22 native detectors pass. The demo's 16 honest players have no finding at all. Demo self-checks failing: 0; fixture self-checks failing: 0.

4. Real-data results

Measured directly, from the committed artifacts. Every rate is among players fpsdet could run on, with a two-sided 95% Wilson interval; under 20 players a rate is not shown.

TF2 league matches with RGL ban labels (tf2-rgl-v2)

[benchmark v1 real data] 9 predeclared baseline draws, fixed 2026-10-06, before any draw of this dataset was run.

What the labels are. RGL's bans are labels, not ground truth: a cheating ban is a league decision about an account, not proof of cheating in any scored match, and never-banned is not proven honest. The positive label (RGL cheating-ban labelled) does not mean that the player cheated in every scored match, or in any particular one. A ban is a league decision, sometimes mirroring another league's.

How much the result depends on the baseline draw. How much does the result depend on which never-banned players build the baseline? In draw k, a never-banned player builds the baseline when the first byte of SHA-256("fpsdet.tf2-baseline-draw/1", NUL, k, NUL, benchmark identity) is under 128, and is scored otherwise. A banned account is always scored. Each draw builds its own baseline cohort from its own baseline half. Nine draws, fixed before any was run. In expectation the range of n independent draws spans the middle (n-1)/(n+1) of the draw-to-draw distribution, 80% for nine, and an odd count makes the median one actual draw.

Draw RGL cheating-ban labelled flagged never-banned comparison flagged Review Watch Queue (RGL cheating-ban labelled) Ratio (95% range) Equal evidence, never-banned comparison Baseline cohort
0 52/182 = 28.6% 51/1,584 = 3.2% 1 124 125 (52) 8.9x (5.3–14.5) 4.7% 1,562 players, c2a7c0549aaa
1 53/182 = 29.1% 44/1,575 = 2.8% 5 109 114 (53) 10.4x (6.2–17.3) 5.1% 1,573 players, ce8efb689d69
2 51/182 = 28.0% 45/1,571 = 2.9% 1 115 116 (51) 9.8x (5.8–16.3) 3.7% 1,577 players, 6618eb1b6e51
3 51/182 = 28.0% 49/1,566 = 3.1% 5 118 123 (51) 9.0x (5.4–14.7) 4.4% 1,579 players, 819a59230e78
4 52/182 = 28.6% 48/1,580 = 3.0% 4 115 119 (52) 9.4x (5.6–15.5) 3.6% 1,566 players, cbd86abae258
5 52/182 = 28.6% 59/1,592 = 3.7% 3 125 128 (52) 7.7x (4.7–12.3) 6.0% 1,554 players, 76f377da1be8
6 51/182 = 28.0% 56/1,567 = 3.6% 10 120 130 (51) 7.8x (4.8–12.7) 5.8% 1,578 players, 724bb0e1fcbb
7 52/182 = 28.6% 51/1,545 = 3.3% 4 117 121 (52) 8.7x (5.2–14.1) 4.9% 1,602 players, 03f59b641c86
8 52/182 = 28.6% 52/1,541 = 3.4% 4 118 122 (52) 8.5x (5.1–13.8) 5.8% 1,608 players, 35f10624068f

The result: the median and the range over all 9 draws. Across all 9 predeclared baseline draws of tf2-rgl-v2, fpsdet flagged (review or watch) a median 28.6% of the RGL cheating-ban labelled accounts it could compare (lowest 28.0%, highest 29.1%) and a median 3.2% of the never-banned comparison (lowest 2.8%, highest 3.7%). The ratio between them was 8.9x at the median, from 7.7x to 10.4x. Every draw is listed in docs/benchmark.md, and none is preferred.

Across the draws Median Lowest Highest
RGL cheating-ban labelled flagged 28.6% 28.0% 29.1%
never-banned comparison flagged 3.2% 2.8% 3.7%
Ratio 8.9x 7.7x 10.4x
Review decisions 4 1 10
Watch decisions 118 109 125
Queue 122 114 130
Equal evidence, never-banned comparison 4.9% 3.6% 6.0%
never-banned comparison compared 1,571 1,541 1,592

What the draws do not vary. [unmeasured] Which 300 extra never-banned players join the comparison is fixed by a public order, not drawn: the 9 draws vary only which half of the never-banned players builds the baseline, so they do not show how much another set of extra players would move the result.

The historical published draw: one draw, not the result. [historical real data] The historical published TF2 result (tf2-rgl-v1) was one baseline draw, chosen by the curator's private key: its ratio, 15.0x, was the most favourable of 6 draws of that data and higher than every draw of this release. It is not the benchmark result. tf2-rgl-v1's own 6 draws ran from 9.9x to 15.0x. Its extra never-banned players were chosen by the curator's private key, not by this release's public order, so its population differs from this release's, not only its baseline draw.

Run What it is Ratio
tf2-rgl-v1, published historical: 1 draw, chosen by a private key, the most favourable of 6 15.0x (one draw)
tf2-rgl-v2 this release: median of 9 public draws 8.9x (range 7.7x–10.4x)

Each detector across the draws. Fires among the players each detector could run on, per draw; a detector's strength estimate (fitted on the development half, checked on the held-out half) can replicate in some draws and not others. Stable decisions do not make a detector robust: this table is per detector.

Detector Status RGL cheating-ban labelled fired (median, range) never-banned comparison fired (median, range) Eligible never-banned comparison Strength ratio (median, range) Replicated in
accuracy measured 4.9% (lowest 3.8%, highest 9.3%) 0.2% (lowest 0.0%, highest 0.3%) 1,541–1,592 46.7 (lowest 20.9, highest 143) 5 of 9 draws
headshot_rate measured 5.0% (lowest 5.0%, highest 10.9%) 0.0% (lowest 0.0%, highest 0.8%) 389–416 26.5 (lowest 8.67, highest 30.6) 3 of 9 draws
extra descriptive_only 2.5% (lowest 2.5%, highest 3.7%) 0.0% (lowest 0.0%, highest 0.0%) 345–366 14.3 (lowest 14, highest 15) 0 of 9 draws
rank_tail measured 25.8% (lowest 20.9%, highest 26.4%) 3.0% (lowest 2.8%, highest 3.5%) 1,541–1,592 8.55 (lowest 5.95, highest 12.7) 9 of 9 draws
supporting_extra descriptive_only 7.1% (lowest 3.6%, highest 14.3%) 0.0% (lowest 0.0%, highest 0.8%) 115–130 no estimate -

What this data cannot show. [unmeasured] 17 of 22 detectors are not observable on it at all: speed, fire_rate, metronome, recoil_floor, mirror, recoil_learned, median_distance, geometry_rate, view_snaps, acquire_timing, account_jump, hidden, quiet_aim, wire, occluded_motion_replay, leftover, voice. The data is per-match totals: shots, hits, sniper headshots and kills per minute. There is no timing, position, view angle or movement, so most detectors are not observable here. Shot times are spread evenly through each match by the converter, so no timing-based detector could mean anything on this data.

CS2 matchmaking (CS2CD) with the dataset authors' labels (cs2cd-v2)

[benchmark v1 real data] One population: the selection has no random draw, and the ids are the dataset's own, so a rebuild gets these exact rows.

Group Scored Review Watch Review or watch, of compared Of all scored
hand-labelled cheater 504 0 14 14/106 = 13.2% (8.0%–21.0%) 14/504 = 2.8% (1.7%–4.6%)
hand-reviewed, not labelled 575 0 0 0/302 = 0.0% (0.0%–1.3%) 0/575 = 0.0% (0.0%–0.7%)
unreviewed match 450 0 4 4/439 = 0.9% (0.4%–2.3%) 4/450 = 0.9% (0.3%–2.3%)

Queue: 18 players, 14 of them hand-labelled cheater (14 of 18, too few to rate); the chance level among compared players is 12.5%. Ratio against hand-reviewed, not labelled: no comparison fires observed; at least 6.4x. Ratio against unreviewed match: 14.5x (3.5–59.0). Every player is one match, so there is no equal-evidence comparison to make here.

5. Eligibility and coverage

For every native detector: whether a controlled fixture proves it, and what each dataset of this release can show (over every draw, for a dataset with draws). Not observable is not the same as not working.

Detector Family Fixture Honest twin tf2-rgl-v2 observable tf2-rgl-v2 status cs2cd-v2 observable cs2cd-v2 status Real challenge telemetry Strength
speed physics yes yes no not_observable yes measured - tf2-rgl-v2: not_calibrated; cs2cd-v2: unsupported
fire_rate weapon_rules yes yes no not_observable no not_observable - tf2-rgl-v2: not_calibrated; cs2cd-v2: not_calibrated
metronome weapon_rules yes yes no not_observable no not_observable - tf2-rgl-v2: not_calibrated; cs2cd-v2: not_calibrated
recoil_floor weapon_rules yes yes no not_observable no not_observable - tf2-rgl-v2: not_calibrated; cs2cd-v2: not_calibrated
mirror weapon_rules yes yes no not_observable no not_observable - tf2-rgl-v2: not_calibrated; cs2cd-v2: not_calibrated
recoil_learned human_baseline yes yes no not_observable no not_observable - tf2-rgl-v2: not_calibrated; cs2cd-v2: not_calibrated
accuracy human_baseline yes yes yes measured yes measured - tf2-rgl-v2: replicated in 5 of 9 draws; tentative in 4 of 9 draws; cs2cd-v2: unsupported
headshot_rate human_baseline yes yes yes measured yes descriptive_only - tf2-rgl-v2: unsupported in 6 of 9 draws; replicated in 3 of 9 draws; cs2cd-v2: unsupported
median_distance human_baseline yes yes no not_observable yes insufficient_sample - tf2-rgl-v2: not_calibrated; cs2cd-v2: unsupported
geometry_rate human_baseline yes yes no not_observable yes insufficient_sample - tf2-rgl-v2: not_calibrated; cs2cd-v2: unsupported
extra human_baseline yes yes yes descriptive_only no not_observable - tf2-rgl-v2: unsupported; cs2cd-v2: not_calibrated
rank_tail human_baseline yes yes yes measured yes measured - tf2-rgl-v2: replicated; cs2cd-v2: replicated
view_snaps human_baseline yes yes no not_observable yes measured - tf2-rgl-v2: not_calibrated; cs2cd-v2: unsupported
acquire_timing human_baseline yes yes no not_observable no not_observable - tf2-rgl-v2: not_calibrated; cs2cd-v2: not_calibrated
supporting_extra human_baseline yes yes yes descriptive_only no not_observable - tf2-rgl-v2: unsupported; cs2cd-v2: not_calibrated
account_jump account_history yes yes no not_observable no not_observable - tf2-rgl-v2: not_calibrated; cs2cd-v2: not_calibrated
hidden information yes yes no not_observable no not_observable - tf2-rgl-v2: not_calibrated; cs2cd-v2: not_calibrated
quiet_aim information yes yes no not_observable no not_observable - tf2-rgl-v2: not_calibrated; cs2cd-v2: not_calibrated
wire information yes yes no not_observable no not_observable - tf2-rgl-v2: not_calibrated; cs2cd-v2: not_calibrated
occluded_motion_replay challenge yes yes no not_observable no not_observable none tf2-rgl-v2: not_calibrated; cs2cd-v2: not_calibrated
leftover relationship yes yes no not_observable no not_observable - tf2-rgl-v2: not_calibrated; cs2cd-v2: not_calibrated
voice relationship yes yes no not_observable no not_observable - tf2-rgl-v2: not_calibrated; cs2cd-v2: not_calibrated

[unmeasured] No real dataset here carries external provider records either, so external evidence has no real-world measurement.

6. Strength and stability

Offline research, never used by the scorer: the label-conditioned evidence ratio fitted on the development half, checked on the untouched evaluation half. For a dataset with draws, see each detector across the draws in section 4; a single draw's estimate is never quoted alone.

Dataset Detector Ratio (95% credible) Stability
tf2-rgl-v2 accuracy median 46.7, from 20.9 to 143 over 9 draws replicated in 5, tentative in 4 of 9
tf2-rgl-v2 headshot_rate median 26.5, from 8.67 to 30.6 over 9 draws replicated in 3, unsupported in 6 of 9
tf2-rgl-v2 extra median 14.3, from 14 to 15 over 9 draws unsupported in 9 of 9
tf2-rgl-v2 rank_tail median 8.55, from 5.95 to 12.7 over 9 draws replicated in 9 of 9
cs2cd-v2 speed 1.09 (4.45e-05–2.68e+04) (no fires in either group: the prior alone, not evidence) unsupported
cs2cd-v2 accuracy 2.89 (0.000119–6.99e+04) (no fires in either group: the prior alone, not evidence) unsupported
cs2cd-v2 rank_tail 43.3 (2.7–3.2e+05) (no comparison fires: the prior sets the far end) replicated
cs2cd-v2 view_snaps 2.89 (0.000119–6.99e+04) (no fires in either group: the prior alone, not evidence) unsupported

7. Challenge qualification

[controlled fixture] Controlled synthetic challenge qualification, not real-world calibration: event-level stand-ins for cheats that know challenges exist, against four challenges in one match.

Scenario Expected Observed Findings Decision
naive reader caught caught 4 review
first probe reader caught caught 1 review
brief reactions not caught not caught 0 insufficient_data
delayed reader caught caught 1 review
last second reader not caught not caught 0 insufficient_data
every second challenge caught caught 2 review
visible enemy cover not caught not caught 0 insufficient_data
radar declared abstained abstained 0 insufficient_data
team share declared abstained abstained 0 insufficient_data
ability declared abstained abstained 0 insufficient_data

8. External-authentication protocol qualification

[controlled fixture] This proves what fpsdet does with a provider's record under each signature state. It says nothing about any vendor detector's accuracy.

Scenario Expected Observed Read as evidence
unsigned unsigned unsigned yes
verified verified verified yes
retired key verified verified yes
invalid invalid invalid no
revoked revoked_key revoked_key no
unsupported algorithm unsupported_algorithm unsupported_algorithm no
unknown key unknown_key unknown_key yes
adapter other provider invalid invalid no
adapter changed field invalid invalid no
self vouching refused refused no

9. Capability matrix

What each technique looks like to the server, what fpsdet claims about it, and the evidence behind the claim, worked out from the fixtures, scenarios and this release's datasets, not declared. Real-data evidence says in how many baseline draws it held. Controlled-only means no public dataset here can show it.

Technique Claim Evidence Detectors Needs
Speed hack detectable controlled; real data observes speed on CS2, with no separation shown speed speed_mps and on_ground on movement samples; a ground-speed cap: expected_max_ground_speed_mps, or weight classes in the profile
Fire-rate modification detectable controlled only fire_rate shot times from the server, not a converter; a weapon rule with its minimum shot interval
Trigger macro detectable controlled only metronome shot times from the server; a weapon rule that is not server-paced
Recoil macro detectable controlled only recoil_floor, recoil_learned recoil_pitch_deg and spray_index on shots; a recoil floor, or enough humans measured on the build
Humanized recoil script partially detectable controlled only mirror, leftover applied_recoil_pitch_deg and compensation_pitch_deg on shots; spray_index
Statistical aimbot partially detectable controlled; real data: TF2 labelled cheaters fire rank_tail, accuracy, headshot_rate more, replicated on one split per baseline draw: rank_tail in 9 of 9, accuracy in 5 of 9, headshot_rate in 3 of 9; CS2 labelled cheaters fire rank_tail more, replicated on one split (which cheat they used is unknown) accuracy, headshot_rate, median_distance, geometry_rate, rank_tail, view_snaps, acquire_timing, extra, supporting_extra hits, and per metric: hitbox, distance, geometry, view turn, acquire time or a declared number; enough humans measured per weapon and rank
Hidden-target tracking detectable controlled only hidden, quiet_aim what each client could see or hear: hidden_track_ms, information_state or vision_state; aim_jitter_deg for quiet aim
Packet or memory reader detectable controlled only wire, occluded_motion_replay wire_error_deg, picture_error_deg and interp_delay_ms on shots; server-planned challenges and challenge_track_ms
Challenge-aware cheat partially detectable controlled only occluded_motion_replay server-planned challenges and challenge_track_ms
Interpolation waiter not detectable architecture statement wire -
Pixel aimbot partially detectable controlled; real data: TF2 labelled cheaters fire rank_tail, accuracy, headshot_rate more, replicated on one split per baseline draw: rank_tail in 9 of 9, accuracy in 5 of 9, headshot_rate in 3 of 9; CS2 labelled cheaters fire rank_tail more, replicated on one split (which cheat they used is unknown) accuracy, headshot_rate, rank_tail hits and hitboxes
DMA frame reader partially detectable controlled; real data: TF2 labelled cheaters fire rank_tail, accuracy, headshot_rate more, replicated on one split per baseline draw: rank_tail in 9 of 9, accuracy in 5 of 9, headshot_rate in 3 of 9; CS2 labelled cheaters fire rank_tail more, replicated on one split (which cheat they used is unknown) accuracy, headshot_rate, rank_tail hits and hitboxes
Wallhack that never aims at hidden enemies not detectable architecture statement hidden, quiet_aim -
External radar sharing partially detectable controlled only voice party_id; enemy_id with what each client could see; a teammate already in review for tracking hidden movers
Account sharing or boosting detectable controlled only account_jump the account's own history on the same weapon and rank

10. Known blind spots

  • [architecture limit] An assist that copies human timing, human noise and human lag.
  • [architecture limit] An aim assist tuned to stay inside the best humans measured: past every human is a finding, better than average is not.
  • [architecture limit] A reader that follows each challenge only briefly, only in the last second of its window, or only beside an enemy the client can see.
  • [architecture limit] A cheat that waits out the interpolation delay.
  • [architecture limit] A capture-card aimbot looking at pixels.
  • [architecture limit] A DMA read of the frame the game actually drew.
  • [architecture limit] A wallhack that never aims at a hidden enemy.
  • [architecture limit] Shared positions acted on no faster than a voice could carry them, or shared by a player who is not in review.
  • [unmeasured] Anything a game's telemetry cannot show. In the benchmark's real data, 17 of 22 detectors on TF2 and 15 of 22 detectors on CS2 are not observable at all, which is not the same as finding nothing.
  • [architecture limit] A number with too few humans behind it: a detector waits for a thick enough baseline, and says so, before it compares.
  • [architecture limit] A server that labels a visible enemy as hidden manufactures cases. That is an emitter bug, and the docs call it out field by field.
  • [architecture limit] External records are read unsigned by default, and a verified signature proves who signed a record, not that its finding is true.

11. Reproduce it

Everything below runs from a clone of the repository with standard-library Python 3.11 or 3.12. The CS2 fetcher and converter also need pip install -r examples/cs2/requirements.txt; the external-signature scenarios need pip install 'fpsdet[auth]'. No step needs a key, an account or anything private.

# Check every committed artifact, offline, without scoring (what CI runs):
PYTHONPATH=src python -m fpsdet benchmark verify
# Rebuild and qualify every controlled fixture and scenario (seconds, no data):
PYTHONPATH=src python -m fpsdet benchmark run --dataset synthetic
# CS2: fetch the one pinned revision (network, about 4.3 GB), rebuild the inputs and run (offline):
PYTHONPATH=src python -m fpsdet benchmark fetch --dataset cs2 --data ~/fpsdet-data/cs2 --python <a Python with the CS2 requirements>
PYTHONPATH=src python -m fpsdet benchmark prepare --dataset cs2 --data ~/fpsdet-data/cs2 --python <the same>
PYTHONPATH=src python -m fpsdet benchmark run --dataset cs2 --data cs2=~/fpsdet-data/cs2
# TF2: fetch the frozen sources (network, rate-limited, hours), rebuild all nine draws and run them (offline):
PYTHONPATH=src python -m fpsdet benchmark fetch --dataset tf2 --data ~/fpsdet-data/tf2
PYTHONPATH=src python -m fpsdet benchmark prepare --dataset tf2 --data ~/fpsdet-data/tf2
PYTHONPATH=src python -m fpsdet benchmark run --dataset tf2 --data tf2=~/fpsdet-data/tf2

prepare checks the download against the frozen source first and says SOURCE_CHANGED, with the digest it expected and the one it found, if upstream moved; it never builds on a changed download as if it were the same. Each run prints MATCH, ENVIRONMENT_ONLY, PROVENANCE_ONLY, PRESENTATION_ONLY, DRIFT, INPUT_CHANGED or NOT_RUN per dataset (per draw, for TF2), with the categories that explain it, and writes every artifact beside the data.

Ordinary CI runs benchmark verify, the synthetic class and the report check on every change. A manual workflow (.github/workflows/benchmark-cs2.yml) rebuilds CS2 from an empty cache at the pinned revision. TF2 is not rebuilt in hosted CI: it takes about 11,000 requests to logs.tf, whose robots.txt asks crawlers to stay off its API, so it is fetched rarely, slowly and by a person.

Formats. Everything above is plain JSON with a declared format:

  • fpsdet.benchmark/1: the definition: datasets, which are historical, archival classes, what the benchmark can and cannot prove.
  • fpsdet.benchmark-dataset/1: one dataset's manifest: source, licence, ids, labels, fetch and prepare commands, frozen selection, pinned inputs, and draws where it has them.
  • fpsdet.benchmark-sources/1: a pinned upstream source list: repository, revision, selection and each file's digest (cs2cd-v2).
  • fpsdet.tf2-sources/1: tf2-rgl-v2's frozen sources as digests and counts: bans, both selections, every selected log; names nobody.
  • fpsdet.cs2cd-fetch-receipt/1: what the pinned CS2 fetcher checked, file by file, at which revision.
  • fpsdet.benchmark-result/1: one run (one draw, for a dataset with draws), every step bound by digest.
  • fpsdet.benchmark-statistics/1: one draw's evaluation statistics without its rows, bound to the evaluation by digest.
  • fpsdet.benchmark-sensitivity/2: every predeclared draw of a dataset, in order, with its own inputs, outputs and numbers, and the median and range across them.
  • fpsdet.benchmark-sensitivity/1: historical: tf2-rgl-v1's published draw beside five public alternative-key draws.
  • fpsdet.benchmark-expected/1: the pinned identity of every result, per draw for a dataset with draws.
  • fpsdet.benchmark-split/1: every subject's development, evaluation or baseline membership (never committed for a dataset with public ids).
  • fpsdet.benchmark-capabilities/1: each technique's claim and the references its evidence is worked out from.
  • fpsdet.benchmark-claims/1: every public statement of the release, with its class and the artifact it comes from.
  • fpsdet.benchmark-release/1: the release itself: every dataset version, the expectations, spreads, capabilities, claims and report, by digest.

12. The historical record

[historical real data] Superseded runs, kept so that what was published can still be checked. Their artifacts verify by digest; none of their numbers is this release's result.

  • tf2-rgl-v1, superseded by tf2-rgl-v2: tf2-rgl-v1 can only be rebuilt with the curator's private pseudonym key, which also chose its one baseline draw. tf2-rgl-v2 is selected by public rules, so anyone rebuilds the same data, and it is run under nine fixed public baseline draws. tf2-rgl-v1 stays as a historical, verifiable record.
  • cs2cd-v1, superseded by cs2cd-v2: cs2cd-v1's fetcher read whatever the dataset's main branch held. cs2cd-v2 fetches one pinned revision, checks every file against a committed source list, and stops rather than fall back. The selection, conversion and labels are cs2cd-v1's. cs2cd-v1 stays as a historical, verifiable record.

How much tf2-rgl-v1 depended on its baseline draw. How much does the result depend on which random half of the never-banned players builds the baseline? The curator's draw is the published one; each other draw uses a public alternative key (SHA-256 of fpsdet.benchmark-alternative-pseudonym-key/{n}, as hex, for draw n). Only the draw changes.

Draw RGL cheating-ban labelled flagged never-banned comparison flagged Queue (RGL cheating-ban labelled) Ratio Equal evidence, never-banned comparison accuracy strength headshot_rate strength extra strength rank_tail strength
curator (published, one draw) 51/182 = 28.0% 30/1,605 = 1.9% 97 (51) 15.0x 2.6% 99 (replicated) 6.99 (unsupported) 25.6 (unsupported) 14.3 (replicated)
1 52/182 = 28.6% 35/1,604 = 2.2% 107 (52) 13.1x 2.9% 124 (replicated) 4.65 (unsupported) 4.68 (unsupported) 12.6 (replicated)
2 51/182 = 28.0% 36/1,604 = 2.2% 102 (51) 12.5x 4.8% 35.6 (replicated) 22 (unsupported) 23.9 (unsupported) 10.3 (replicated)
3 52/182 = 28.6% 45/1,557 = 2.9% 121 (52) 9.9x 4.8% 34.2 (replicated) 19.6 (unsupported) 10.8 (unsupported) 7.9 (replicated)
4 52/182 = 28.6% 33/1,521 = 2.2% 106 (52) 13.2x 4.3% 23.2 (unstable) 19.5 (unsupported) 12.1 (unsupported) 12 (replicated)
5 52/182 = 28.6% 37/1,553 = 2.4% 108 (52) 12.0x 3.8% 30.1 (replicated) 20.1 (unsupported) 28.3 (unsupported) 11.6 (replicated)

The published draw's ratio, 15.0x, ranks 1 of 6 draws (from 9.9x to 15.0x): one draw of a random split, not a fixed property of fpsdet, and not this release's result.

Third-party systems

Nothing but fpsdet is evaluated here. A future version can evaluate another system on the same frozen populations without forcing it into fpsdet's schema. It would take one neutral record per subject:

{"format": "fpsdet.benchmark-submission/1", "dataset": "tf2-rgl-v2", "subject": "an id from your own rebuild", "decision": "flag | no_flag | abstain", "observation_kinds": ["optional, the system's own names"], "confidence": null}

It would be scored with the same label semantics, splits, eligibility rules (abstain is not a miss) and intervals, and never folded into one number. Not implemented: the immediate goal is that fpsdet itself can be reproduced and checked.

Provenance

  • synthetic-v1: result sha256:98b0d3bf54585f9a8ee3acd1c8bc72bd7ec326695b82b83a19f93b035c320414, manifest sha256:6d091db274e57c14f15a0e74248260c2466c131b85497fee62d01d8094d565eb, detector sha256:a75b28fb3b00fa5868444318d44c7c4d235e94a76a85a96fad4f1f040b80ad81
    • pinned on Python 3.12.3 (Linux, x86_64); timings, not part of the identity: build 6.18s, qualify 0.56s
  • tf2-rgl-v2: spread sha256:485d2720a2dc1be3a25101fe01b5027949715ed72993b22a2a60687f9d7710b2 over 9 draws, manifest sha256:40277b22048f175eab04e7a1a4bea9b2a6bb8fb8aeae4a700240d928f0fdd4ae
    • draw 0: result sha256:43a1e53119301f7eea0b757bdad1d09bc7be43c0e8a610410569936f79a1eef3; inputs: events sha256:81926ccd8b7ba1e59e875da33194d52fb69aedb8556f3e92f4aadb805fe4901d, labels sha256:a8180ab52620ea1cd1ef3204c4bb33b7532896a9a36a962bdbe10828fd244094, cohort sha256:c2a7c0549aaa5a6bdea4eca26939e4fd114127a597b7182b4c10f81b1302f147; evaluation sha256:da6a8d0ff5570872ea3dce1daa3bf7fe0b7eea70d1ade0ef1251a58b3acd4017; strength sha256:fbbf117f02d9090c8aa1a622ea3241856caccea1eb0cb89338ffcb65f2f6bacc
    • draw 1: result sha256:51eba4dc0621d33b479acfa519c6ef19002d53fced127f0d85ee5963493dc061; inputs: events sha256:c9d66bb6fbf32114922318d5d719045c4e6d48b8a9c859267fc27b0d02f60354, labels sha256:b15d8c409c992da5ef56e1375392ac2667486003e8a6d2a26ffb1e786901fe99, cohort sha256:ce8efb689d699fc252254a384b8c25bcdc43deab976d08406fb87d91bbe3bab4; evaluation sha256:57068896f50c50042972cbcf72bd2f50e4533cafd636ba5ac16f40babd6d1671; strength sha256:383c3256ecea7c8e241058f9f2d070db3076ba95c1300b43454ec545f08727d0
    • draw 2: result sha256:dc711225171609a9f1c5e1ac45b6be06b5f819cfca951316f22ac273e498c9a9; inputs: events sha256:fc863b569de35c9c226a74d3c26fb9f687c081a139c85513f6ce3de1bddbc6f2, labels sha256:163aaa69b78c2a8ea2c9cc94939e433d2dd3e96763726cf7605a90504d4d38f2, cohort sha256:6618eb1b6e51b57e365411161620e2ecac22372686210218053e6812e269d1af; evaluation sha256:f16f0293a61e537c973b1d53ddda5271c10d8de694fe9a8e8d8e5b81281bb3b6; strength sha256:c09af2ab9dcf13b79969cccb77edd7d522f9fd03aa0a21d10dbb82df93ad6001
    • draw 3: result sha256:7ae3add0b7c36530706766f96e144fa5ca59c8237f8aa520f3237c1818f7d24b; inputs: events sha256:79b01251f8a26d89a3fc69e15c82661705e6cc41785ba77af250d3d9e5d766fd, labels sha256:974fd13d04946cead08d94a363cfb76addb09185d76a6f364d25a8db174cbb6a, cohort sha256:819a59230e78bf85d99fea34d0eacd40a0b15f4d57d5c30daf88a1c972f0b3c5; evaluation sha256:aec926516b8c3ef3034a0e6eebdbc901c8c54c724bdd57599d75fb70c50ad4e2; strength sha256:09a466ead6bd02e71aebe312952eb4271c812caa92429a791a56e9d390fe7e3b
    • draw 4: result sha256:e3f239637fc38e930af2c40ad555ac521a407682e54081aae8bef041a141a114; inputs: events sha256:303285b7ed0abf3a6fc25adfc9639b4dddd3e3f925db2ccb861ec3ad4e80c248, labels sha256:a6a7dbb253722f765e19c29ba8223b64292b60da42e399e69b12e2032c0acd4a, cohort sha256:cbd86abae25879e4634a098ffcf520116430bf6d31bcd578a68812bd80393a9e; evaluation sha256:d6c521c66bc0afd1792ad08cd23fe91d78d491c185cdbc7d12622c5985ac6322; strength sha256:035f5a72db19d99eaa546ea94e2f91d6b05816255196e59c59cbf6eb0dc5edb2
    • draw 5: result sha256:0051cfeaf0411f8a30192d5cfc8d4b247fc605524bf0d27477983ef356948250; inputs: events sha256:8023b2bd3c9f1926a2291b6c3c7082496f2dec940bcbc5317dee1f28f1def28f, labels sha256:802fbbc1b1b4f3077cc462855c8ccd9e6d88a7e985e3aaecf71ab923c8318631, cohort sha256:76f377da1be88bbd444f91f095ba875af202dd563269dcb39cbe49d38bfebf8b; evaluation sha256:6c89427f83cb953259068791a3eb6fa8ff0e7da5b5f86c3f75003d273d28b621; strength sha256:33f9d3b8497a401fe81407cb157eb4ba1209c8473399453688b6599d83e2d3af
    • draw 6: result sha256:4b8cb104353a90a8cbc0fd775ea8b3cbf534727122adc7e6834fa4627c486270; inputs: events sha256:0209952bdbbc3bd9ae3d13fc4a2f6a97a09189f722a23c0c983b55c91f8e21c6, labels sha256:4a7eac11986c981a157674830ed0312cad09b8fccbf9a5763ded0e7e5671549c, cohort sha256:724bb0e1fcbbf09514230893231d09abcfdb8635be89ff0b0341b0b6fd9cea63; evaluation sha256:9dbedebebf474299ae82c1fd2142810fcb0f5748c8c5e0fed952c39f63cda118; strength sha256:b7064f1bc53975008000612e6c0888af398d9146b61420734e03f4e32ab93f48
    • draw 7: result sha256:8ed175285d0538231c327ddf96f620dc74eec71c8b811e0af626cc75b11d43e5; inputs: events sha256:fc6a678a4d5b5ea98d261a210db02d20a3d3f90332496c7b3f6c641cfa66302c, labels sha256:90a3fef2350adf4d2e1ee2b860688e7d23c61c27294abf7949af308b57d5d673, cohort sha256:03f59b641c86fee4667fc1f9a5d7502e2883aef3709e8dda29787b0f6e7a1caa; evaluation sha256:23cd95bf9bf1b7c6105f5ae6c4d4b7e3d6ca5a55567cb14b2e67e59808bfc4d5; strength sha256:9fdc7818ad23637d31928aaa7f1615e59ae3c455bf197b873167d5ff584b5804
    • draw 8: result sha256:03ac5919a4c503f109d6155cdf3ed89f550a35aff9cb960b6a3491c5a8e31c9a; inputs: events sha256:a6eb0998b9cf912127cba70d74a1149b3d817012893723b8ac69aec323fb3b9d, labels sha256:821e97d814069c78a6059f924c9589ab54da759f927d6c1dd32fe8ccec346cec, cohort sha256:35f10624068f8dbd3c625fb231033a7dc1a351f57f7bd33dab043663f330d86a; evaluation sha256:0b30be7f3d19b6dc1c06dc053ce73eb0e44de7675db8b83adeba7021a9f0addb; strength sha256:32757cb7f469aa798d1c808ff52607647e3b190a849ceb975205e3901f8d53e6
  • cs2cd-v2: result sha256:e840edb0b7c96b3fb463831914f5a07719e04e067ac0beb5ad46787985b3f913, manifest sha256:76af5b2e292921c5f44a0a224947605f623219b507b24a15a39e126fb9c8fb82, detector sha256:a75b28fb3b00fa5868444318d44c7c4d235e94a76a85a96fad4f1f040b80ad81
    • inputs: events sha256:3fc90f3708712dd7684cffc192358d82fa6d3da52934a799eed27af7f386e537, labels sha256:582220fa0489ba421a73f51b8347772a776f827fb3f2801fefa13f6a7c8a1930, cohort sha256:143ce6f0e26bd4df6bfcde2f3f7ea84d96245aea321e982d380e8b1d68108fa6; packets sha256:1fddbfedecaf603807a67a681cb0d4f85a6c3f35abfe6a9ded6155e93a29fa44; evaluation sha256:e83d92971ff3df8032ab788e69bc3bce42ebdb76b889cbb06e0a9923670aa291; strength sha256:31c7a2c80465d37f5902c83184e980d0850c5fc5be91cc8b5150c4b07185ecf6; split sha256:ce198f24856b735c9b6196cc28c7417026922b3b71f1eab35b3a78b669422d2d
    • pinned on Python 3.12.3 (Linux, x86_64); timings, not part of the identity: verify_inputs 0.47s, score 44.53s, verify_packets 0.14s, evaluate 10.2s, strength 0.15s
  • tf2-rgl-v1: result sha256:bb41b1404bd4198d278e42d7d083a02e1ed6b8c6b4aa5f7ec4a0b0792744bd7b, manifest sha256:b7b094a2863b2033fe855cbd298cfcc3e1609b8259eb32140e7d66d4267c121b, detector sha256:a75b28fb3b00fa5868444318d44c7c4d235e94a76a85a96fad4f1f040b80ad81
    • inputs: events sha256:27d43fa684f92948b97c0a8d44b83b5cb157f005a634343951f14f0ac3350af1, labels sha256:1981f3c068cc5f8b64afde3f8bbdfa17ee3cff9035250ed8fb76045011e7c3ec, cohort sha256:fc38c02f30b9ccda4b922d8806e9ab0fb50569ebd020ea653a8582e06017fd35; packets sha256:33894907a9b45162b75709891dd2fd311196d23244068c2e3ce3f79e7b3494e6; evaluation sha256:5a0dfabd2ba984bdf6065103fb7c7231146774f8efc83239154c1be46610d612; strength sha256:3c1fe2bc0e417c708dc86cb07e5868de396d58e1a704eb65df7b655eceb43f7f; split sha256:7088e97efaa23e7aab6b19d6b4e80b032a889d477f3c3c57a12857250d950274
    • pinned on Python 3.12.3 (Linux, x86_64); timings, not part of the identity: verify_inputs 0.63s, score 44.41s, verify_packets 0.28s, evaluate 8.66s, strength 0.18s
  • cs2cd-v1: result sha256:88263520c366c1cc23c40e75cec1db3b47df7ab983b55fc03fa678739a85248e, manifest sha256:1f093b1587215a0451198e0361c1c095f2ea2b038352be709785b77dd205b2db, detector sha256:a75b28fb3b00fa5868444318d44c7c4d235e94a76a85a96fad4f1f040b80ad81
    • inputs: events sha256:3fc90f3708712dd7684cffc192358d82fa6d3da52934a799eed27af7f386e537, labels sha256:582220fa0489ba421a73f51b8347772a776f827fb3f2801fefa13f6a7c8a1930, cohort sha256:143ce6f0e26bd4df6bfcde2f3f7ea84d96245aea321e982d380e8b1d68108fa6; packets sha256:1fddbfedecaf603807a67a681cb0d4f85a6c3f35abfe6a9ded6155e93a29fa44; evaluation sha256:e83d92971ff3df8032ab788e69bc3bce42ebdb76b889cbb06e0a9923670aa291; strength sha256:31c7a2c80465d37f5902c83184e980d0850c5fc5be91cc8b5150c4b07185ecf6; split sha256:043a0ce26ea176e435305e10d40c5561071333d55a116f5262818bfb61a367a2
    • pinned on Python 3.12.3 (Linux, x86_64); timings, not part of the identity: verify_inputs 0.44s, score 42.24s, verify_packets 0.14s, evaluate 10.32s, strength 0.15s