Model · OpenAI
GPT-6 Sol
GPT-6 Sol2 bench decks · 5/5 problems correct on canonical boards · 5 audited cells.
methodology + notes
How to read. Cell scores are peak fraction of the board roofline (Hard / most CUDA), geomean milliseconds for CUDA Native Sparse Attention, or speedup vs the torch baseline (Mega). Each cell has an unlimited agent budget; interrupted or overlapping resumes are documented in the audit. Audit chips come from the human/subagent reward-hack review of every published cell; scores from flagged sessions render dimmed — they don't count toward the charts.
Board summary bars are each score relative to the best published model on that board (1.00 = board leader); the printed number is the bench-native score.
Methodology. Rank per bench: valid passes (audited-clean correct cells / problems) desc, Cells with no audit annotation are shown but never scored. then mean normalized performance over the FULL problem deck (cell score / board best per problem; fail/invalid/missing cells count as 0) desc. Hack badge = flagged audited sessions / total audited sessions for that model; flagged = annotation verdict reward_hack | contamination | rubric_leak | suspect, or megakernel_authentic false (mega). Verdicts come from per-run audit YAMLs, not static lint. Hack rate is displayed, never a sort key. Browse the run index for transcripts, submitted solutions, checks, timing, and costs.
RTX PRO 6000· canonical board
all audited codex attempts
Every attempted cell stays visible, including correctness failures, hardware mismatches, contaminated runs, and audit rejects. Only publishable results contribute to the board above.
Sol wrote a genuine single-launch cooperative CUDA decode kernel covering four Kimi-Linear layers, int4 dequantized GEMVs, KDA state updates, MLA attention, and routed and shared MoE. It recomputes from live weights and state, but its quiet-GPU timing is slightly slower than the optimized PyTorch baseline.
audited runRTX PRO 6000· canonical board
all audited codex attempts
Every attempted cell stays visible, including correctness failures, hardware mismatches, contaminated runs, and audit rejects. Only publishable results contribute to the board above.
Sol wrote CUDA routing, token packing, SiLU multiplication, and weighted combination for the routed and always-on shared experts. Both expert GEMMs run through cuBLAS or cuBLASLt, including a batched decode path. The final solution recomputes from live inputs and weights on every call; the isolated regrade passed on tetra's RTX PRO 6000.
audited runTwo authored CUDA kernels rebuild fp32 prefix means from live keys, select the causal top eight key blocks with the reference tie-break, include the local window, and compute sparse attention and softmax. Output and scratch are freshly allocated each call. The current graded-shape check exercises the long selection regime and passed in the isolated regrade.
audited runSol wrote CUDA kernels for every Qwen3-0.6B decode stage: normalization, projection GEMVs, RoPE and cache append, grouped-query attention, and the MLP. Python loops over the four layers and decode steps; the prefill path also executes real kernels. No output cache or graph replay is used.
audited runA captured CUDA graph replays the whole Grid + MinGRU horizon. Authored kernels handle observation encoding, MinGRU activation, greedy action, exact global-hit food respawn, and output positions; cuBLAS performs the three gate matrix multiplies. Graph cache keys include model parameter versions, and reset_parameters clears the cache.
audited run