A replay-verified benchmark for AI decision models. First game: Mortal Kombat II (Genesis). Liu Kang fights 15 VeryHard CPU opponents. Models read game memory, not pixels.

Edition mk2-liukang-v1 · prompt SHA-256 loading

Policy Table

The model answers 8 situation questions once per fight, before play. Rank: mean damage dealt (of 120) over 15 fights x 10 repeats. Cost and call latency are self-reported.

By opponent

Mean damage dealt to each CPU opponent, official rows only. Brighter is more damage.

Replay

Every number above comes from a replay like this one.

Clef-flash beats the VeryHard Shang Tsung with 12 health left

Clef-flash beats the VeryHard Shang Tsung with 12 health left. Pre-edition demo, replayed frame for frame from the sim.

Clef answered 8 situation questions in one call before the round. Code reads game memory each frame and plays a move from Clef's probabilities.

Wins are rare. That is why the rank is damage dealt, not wins.

Live

The model answers the current situation every 0.12 s, at most 3 requests in flight. Rank: replayed mean damage dealt. Network latency is self-reported and is not a rank key.

Method

  • Same save states, same 17 moves, same prompt, same seeds for every row.
  • Rank is mean damage dealt, high to low. Wins are reported, not ranked.
  • script is the rules-bot floor. random picks uniformly from the 17 moves. Both call no model, so they cost $0.
  • A new model version or track is a new row. A new prompt or temperature is a new edition.
  • Replay re-runs the game from the submitted answers and checks the outcome. It does not re-call the model, so the model name, cost, and latency are attested by the submitter.
  • Survival is the mean round length in seconds. Top moves is the share of decisions per move.
  • Smoke rows (one fight, one round) are grey and sit below official rows.
  • This page renders board.json. A GitHub Action rebuilds it from results/verified/ on every merge to main.

Submit

Bring your own ROM. Nobody ships a ROM.

uv run mk2_clef.py --rom <your file>                     # install your ROM
uv run fightbench.py table --policy my.py:MyPolicy        # or: live --policy my.py:MyPolicy
git add submissions/ && git commit && gh pr create        # commit only your submissions/ file

Write a Policy class (see fightbench.py). Set usage to report tokens and $; the harness times each call. The maintainer runs uv run fightbench.py replay pending on the PR and merges. The board updates itself.