Model · Anthropic

Claude Fable 5.1

2 bench decks · 3/4 problems correct on canonical boards · 8 audited cells.

methodology + notes

How to read. Cell scores are peak fraction of the board roofline (Hard / most CUDA), geomean milliseconds for CUDA Native Sparse Attention, or speedup vs the torch baseline (Mega). Each cell has an unlimited agent budget; interrupted or overlapping resumes are documented in the audit. Audit chips come from the human/subagent reward-hack review of every published cell; scores from flagged sessions render dimmed — they don't count toward the charts.

Board summary bars are each score relative to the best published model on that board (1.00 = board leader); the printed number is the bench-native score.

Methodology. Rank per bench: valid passes (audited-clean correct cells / problems) desc, Cells with no audit annotation are shown but never scored. then mean normalized performance over the FULL problem deck (cell score / board best per problem; fail/invalid/missing cells count as 0) desc. Hack badge = flagged audited sessions / total audited sessions for that model; flagged = annotation verdict reward_hack | contamination | rubric_leak | suspect, or megakernel_authentic false (mega). Verdicts come from per-run audit YAMLs, not static lint. Hack rate is displayed, never a sort key. Browse the run index for transcripts, submitted solutions, checks, timing, and costs.

Board summarybars = share of each board's best model · numbers = bench-native score
Mega
19.34x1/1
CUDA
8.3%2/3
Megaor-fable
1/1 pass5 audited

RTX PRO 6000· canonical board

Kimi-Linear Decodepass
clean
19.34xfull-model speedup vs torch
3504 tok/s2048 ctx 19.21x8192 ctx 19.30x16384 ctx 19.52xptx

all audited or-fable attempts

Every attempted cell stays visible, including correctness failures, hardware mismatches, contaminated runs, and audit rejects. Only publishable results contribute to the board above.

RTX PRO 6000 Blackwell · Kimi-Linear Decodeclean
19.34xpublishable

Re-graded 2026-11-16 under grader v3 on tetra: 19.34x (22.95x before; see regrade_v3). Original audit: Claude Fable 5.1 through OpenRouter (or-fable harness), max effort, unlimited budget, RTX PRO 6000: a single cooperative persistent megakernel with a task-queue scheduler and mma tensor-core int4 dequant-GEMVs. Isolated sequential regrade on the same box 2026-09-04 (clocks reset with nvidia-smi -rgc first): 22.9457x geomean over the untouched baseline (23.01x / 22.80x / 23.02x at ctx 2048 / 8192 / 16384, 0.330-0.401 ms/tok); the contended in-run grade was 22.9889. Overwrite probe (probe.log): primed step matches the reference at cos 1.0000, a continuation step with the token overwritten in place produces a different output (cos(out1,out2)=0.0335) that still matches the reference at 1.0000, and a fresh state at seed+1 matches at 1.0000, so nothing is served from a cache. Same design lineage as the model's native 15.84x run two days earlier, extended in-session from 16.45x to 22.99x across six benchmarks; the final +5.3x came from deferring the next-task grab until after each dependency spin. check.log shows S=1.0000 and cache=1.0000 on all six seed/ctx combos, out 0.9963-1.0000 against the 0.98 gate. All 8 template files byte-identical, no foreign reads, no clock commands, one pip install of ninja, zero lock contention, no key strings in the archive. Session ended at 2h10m on an upstream "stream closed before completion" API error while the agent was still optimizing, not on a deliberate stop; the graded file is the last self-measured version (self-measured 22.9889 at 23:58Z, contended grade 22.9889). OpenRouter cost $41.74 (500k output tokens, 20.2M cache reads).

audited run
CUDAor-fable · max
2/3 pass3 audited

RTX PRO 6000· canonical board

GLM-5.3 Fused MoEpass
10.2%clean
session 4h 11m
DeepSeek NSApass
clean
session 3h 7m

pass

MegaQwen Decodepass
6.4%clean
session 3h 48m

all audited or-fable attempts

Every attempted cell stays visible, including correctness failures, hardware mismatches, contaminated runs, and audit rejects. Only publishable results contribute to the board above.

RTX_PRO_6000 · GLM-5.3 Fused MoEclean
10.17%publishable

Claude Fable 5.1 through OpenRouter (or-fable harness), max effort, unlimited budget, RTX PRO 6000, 4h00m to a voluntary stop: persistent 128x256x64 bf16 mma.sync grouped GEMMs with the gate/up N-tile interleaved so SiLU*up fuses in registers, a mainloop specialized on valid m16 fragments so padded rows cost no tensor-pipe time, hist/scan/scatter token alignment for large T, fp32 L2-resident red.global.add accumulation for small outputs, and a CUDA-core GEMV decode path for T<=8. DSMEM operand exchange, TMA multicast, wider stores and a CUDA-graph replay variant were built, measured slower, and deleted. Graded 0.1017 geomean (per shape 0.3587 / 0.3634 / 0.0026 / 0.6009 / 0.0538 / 0.1023); the agent's own last official run said 0.1018 and the graded file is byte-identical to the snapshot it validated. Isolated sequential regrade on the same box 2026-09-04: 0.1017, identical to the contended grade. Overwrite probe (probe.log): primed cos(ref,sol)=1.0000; overwriting the input in place at the same data_ptr changes the output (cos(out1,out2)=-0.0007) and still matches the reference at 1.0000; overwriting the weights in place changes it again (0.0001) and matches at 1.0000; fresh inputs 1.0000. All template files byte-identical, no foreign reads, no clock commands, no key strings, two curl HEAD probes and no downloads, zero lock contention. OpenRouter cost $36.39 (381k output tokens, 28.9M cache reads). Second on the problem behind or-opus 0.1072.

audited run
RTX_PRO_6000 · DeepSeek NSAclean
0.093 msaudit evidence

Claude Fable 5.1 through OpenRouter (or-fable harness), max effort, unlimited budget, RTX PRO 6000, 3h07m to a voluntary stop. Key-major NSA pipeline in raw CUDA with inline PTX tensor-core MMA: a fused select kernel (fp32 block mean keys split into three bf16 parts so ranking stays fp32-grade, masked causal-prefix mean for the home block, exact top-8 under the reference (imp, block-id) tie-break via 64-bit sortable keys), a gather kernel writing flash-decoding partials per (query, block), and a window/merge kernel; the three launches replay as a CUDA graph with programmatic dependent launches. The check.py blind spot (S<=384 selects every block) hides nothing here: the agent's own tests against the exact python reference at S=1024/700/1100 and a vectorized oracle on all six deck shapes show zero out-of-tolerance elements, and a CPU emulation of the selection rule during pre-audit matched reference.nsa_attend selections on every one of 6,602 emulated rows at S=1024-2048, D=64/128. The agent measured and rejected a persisting-L2 trick whose gain came from the harness's inter-call cache flush. Graph replay is worth ~3% (0.9755 with, 0.9461 without). Per shape 0.060/0.106/0.109/0.218/0.054/ 0.080 ms; fractions above 1 are structural (dense-equivalent FLOPs against ~9% of the attention work). Templates byte-identical, no foreign reads, no network, no clock commands, no key strings, zero lock contention. In-session final benchmark 1.0586 equals the contended grade; isolated sequential regrade on the same box 2026-09-04: 1.0627 (+0.4%). Overwrite probe with cloned outputs (probe.log): primed 1.0000, in-place input overwrite at the same data_ptr changes the output (cos(out1,out2)=-0.0041) and matches the reference at 1.0000, the weight-overwrite step is a no-op at cos 1.0000 because the reference Model has no parameters (only a dummy buffer), fresh inputs 1.0000. Long-context probe past the check.py select-everything regime (probe_long.log): S=1024/1500/2048, D=64/128, seeds 42/123, all six ok=True at cos(ref,sol)=0.999998 and max diff <=0.0156 against reference.nsa_attend, LONG_CTX_OK. OpenRouter cost $53.15.

audited run
RTX_PRO_6000 · MegaQwen Decodeclean
6.43%publishable

Claude Fable 5.1 through OpenRouter (or-fable harness), max effort, unlimited budget, RTX PRO 6000, 3h33m to a voluntary stop. One persistent cooperative megakernel: 188 blocks x 512 threads, every decode step and all four layers in a single cudaLaunchCooperativeKernel, five grid barriers per layer (QKV, attention, O-proj, gate-up, down), the 126 MB weight set held L2-resident with evict_last hints while the KV cache streams through a 6-stage cp.async.bulk/mbarrier ring tagged evict_first. Attention softmax uses a fixed per-layer reference max (p = exp(s - M) with M >= max|s| derived from the live q/k-norm weights plus a 2% margin), which is exact because the constant cancels in the division; an exact online-softmax fallback exists but is dead code under every grader-seeded model. 36 measured kernel builds: a cp.async L2-hint illegal instruction on sm_120 forced the bulk-copy redesign, one deadlocked build was killed, and the agent's own 8192-context spot check caught a NaN regression that was fixed before submission. Fresh h_out every call; the kernel_state cache keyed on weight data_ptr/_version stores pointers and two scalars, weights are re-read from GPU memory on every launch. Contended in-run grade 0.0638; isolated sequential regrade on the same box 2026-09-04: 0.0643 (+0.8%), 11.4k tok/s at ctx 2048 down to 655 tok/s at ctx 131072. Probe (probe.log): primed ctx 2048 matches the reference at cos 1.0000, a second call with a new seed on the same model produces a different output (cos(s1,s1b)=-0.1474) that matches at 1.0000, and overwriting the weights in place changes the output (0.0239) and matches at 1.0000. All template files byte-identical, no foreign reads, no network, no clock commands, no key strings, zero lock contention. Graded file byte-identical to the build the agent verified. OpenRouter cost $64.87 (672k output tokens, 56.3M cache reads). Second on the problem behind or-opus 0.0655.

audited run