Model · Anthropic
Claude Haiku 5.5
Claude Haiku 5.52 bench decks · 4/5 problems correct on canonical boards · 5 audited cells.
methodology + notes
How to read. Cell scores are peak fraction of the board roofline (Hard / most CUDA), geomean milliseconds for CUDA Native Sparse Attention, or speedup vs the torch baseline (Mega). Each cell has an unlimited agent budget; interrupted or overlapping resumes are documented in the audit. Audit chips come from the human/subagent reward-hack review of every published cell; scores from flagged sessions render dimmed — they don't count toward the charts.
Board summary bars are each score relative to the best published model on that board (1.00 = board leader); the printed number is the bench-native score.
Methodology. Rank per bench: valid passes (audited-clean correct cells / problems) desc, Cells with no audit annotation are shown but never scored. then mean normalized performance over the FULL problem deck (cell score / board best per problem; fail/invalid/missing cells count as 0) desc. Hack badge = flagged audited sessions / total audited sessions for that model; flagged = annotation verdict reward_hack | contamination | rubric_leak | suspect, or megakernel_authentic false (mega). Verdicts come from per-run audit YAMLs, not static lint. Hack rate is displayed, never a sort key. Browse the run index for transcripts, submitted solutions, checks, timing, and costs.
RTX PRO 6000· canonical board
all audited selected attempts
Every attempted cell stays visible, including correctness failures, hardware mismatches, contaminated runs, and audit rejects. Only publishable results contribute to the board above.
No solution. In two legs totalling 93 minutes Haiku 5.5 at --effort max never wrote a kernel: the first leg ran on a 200,000-token window because Claude Code 2.1.292 did not recognize the model id, hit the 128,000-token output cap four times and was stopped by Claude Code's autocompact-thrash guard; the resumed leg on 2.1.294 spent four turns of pure reasoning, each hitting the 128,000-token output cap, and the fourth ended the session with an API error.
audited runRTX PRO 6000· canonical board
all audited claude attempts
Every attempted cell stays visible, including correctness failures, hardware mismatches, contaminated runs, and audit rejects. Only publishable results contribute to the board above.
Haiku 5.5 wrote the GLM-5.2 MoE layer as four hand-written CUDA kernels: a one-block counting sort of the routed pairs with the shared expert scheduled as one more expert group, a grouped bf16 mma.sync GEMM for gate/up with silu(gate)*up fused into its epilogue, a second grouped GEMM for down, and a per-token reduce that applies the routing weights and adds the shared expert.
audited runHaiku 5.5 wrote exact per-token NSA selection and attention as two plain fp32 CUDA kernels, one warp per query: fp32 block key sums, block scores as q·sum/64 with the diagonal block scored as the causal mean of its real q·k scores, a warp-shuffle top-8 with ties to the larger block index, then one-pass softmax over the eight selected blocks plus the 64-token window with overlap removed.
audited runHaiku 5.5 wrote the four-layer Qwen3-0.6B-geometry decode as a host loop of eight plain CUDA kernels per step: noise mix, then per layer a fused RMSNorm plus QKV warp-GEMV, per-head Q/K norm and RoPE with the KV-cache append, split-KV GQA attention over the full cache with a separate combine, the O projection plus residual, fused RMSNorm plus gate/up SiLU, and the down projection.
audited runHaiku 5.5 runs the rollout as four fp32 CUDA kernels per step: layer 0 collapsed to a 4-input map (W_gru[0]·W_enc folded once per call), MinGRU layers 1 and 2 as cp.async double-buffered SIMT GEMMs over 128-env tiles with the gate math in the epilogue, the action head as per-slice partial logits, and a head kernel that does argmax, the move, reward and food respawn with a closed-form LCG jump.
audited run