Model · Anthropic
Claude Sonnet 5.5
Claude Sonnet 5.52 bench decks · 5/5 problems correct on canonical boards · 5 audited cells.
methodology + notes
How to read. Cell scores are peak fraction of the board roofline (Hard / most CUDA), geomean milliseconds for CUDA Native Sparse Attention, or speedup vs the torch baseline (Mega). Each cell has an unlimited agent budget; interrupted or overlapping resumes are documented in the audit. Audit chips come from the human/subagent reward-hack review of every published cell; scores from flagged sessions render dimmed — they don't count toward the charts.
Board summary bars are each score relative to the best published model on that board (1.00 = board leader); the printed number is the bench-native score.
Methodology. Rank per bench: valid passes (audited-clean correct cells / problems) desc, Cells with no audit annotation are shown but never scored. then mean normalized performance over the FULL problem deck (cell score / board best per problem; fail/invalid/missing cells count as 0) desc. Hack badge = flagged audited sessions / total audited sessions for that model; flagged = annotation verdict reward_hack | contamination | rubric_leak | suspect, or megakernel_authentic false (mega). Verdicts come from per-run audit YAMLs, not static lint. Hack rate is displayed, never a sort key. Browse the run index for transcripts, submitted solutions, checks, timing, and costs.
RTX PRO 6000· canonical board
all audited claude attempts
Every attempted cell stays visible, including correctness failures, hardware mismatches, contaminated runs, and audit rejects. Only publishable results contribute to the board above.
Sonnet 5.5 built a genuine single-launch megakernel for the [KDA, KDA, KDA, MLA] motif: 188 persistent CTAs walk 26 dependent phases per token behind grid barriers, dequantizing int4 weights in registers into mma tensor-core GEMVs, running the KDA delta-rule state updates, absorbed MLA attention over every cached token with an in-place cache append, and the router, top-8 and nine-expert MoE per layer.
audited runRTX PRO 6000· canonical board
all audited claude attempts
Every attempted cell stays visible, including correctness failures, hardware mismatches, contaminated runs, and audit rejects. Only publishable results contribute to the board above.
Sonnet 5.5 wrote the GLM-5.2 MoE layer as hand-written CUDA: a device-side counting sort of the routed pairs, two swap-AB bf16 tensor-core grouped GEMMs (silu(gate)*up fused into the first epilogue, routing weights and fp32 atomics into an L2-resident accumulator in the second), the shared expert scheduled as one more expert, and a separate cooperative GEMV kernel for the single-token decode shape.
audited runSonnet 5.5 built exact per-token NSA block selection (fp32-exact block key sums split into three bf16 pieces, tensor-core importance, a causal mean on the diagonal block, top-8 per row) and a key-stationary attention kernel that stores each (row, block) partial output as int8 with a per-row scale before a softmax-weighted merge. Short D=64 shapes use a fused row-stationary kernel with no int8 step.
audited runSonnet 5.5 retargeted the MegaQwen idea to one cooperative persistent kernel per decode call: 176 compute CTAs run RMSNorm plus QKV, Q/K norm plus RoPE plus split-KV causal GQA over the full cache, the O projection and the SwiGLU MLP for all four layers behind a hand-rolled grid barrier, while 12 mover CTAs prefetch upcoming KV rows and streamed weights into L2.
audited runSonnet 5.5 fused the whole rollout into one persistent kernel: MinGRU layers 1-2 on fp16 tensor cores with a 3-term hi/lo operand split for near-fp32 accuracy, layer 0 as a 4-input fp32 map, heads, env step and LCG respawn in-kernel, and a host loop that relaunches only if its assumed batch-wide food-hit pattern was wrong, which reproduces the reference's global RNG advance exactly.
audited run