Model · Google
Gemini 4 Neon (XHigh)
2 bench decks · 4/4 problems correct on canonical boards · 5 audited cells.
methodology + notes
How to read. Cell scores are peak fraction of the board roofline (Hard / most CUDA), geomean milliseconds for CUDA Native Sparse Attention, or speedup vs the torch baseline (Mega). Each cell has an unlimited agent budget; interrupted or overlapping resumes are documented in the audit. Audit chips come from the human/subagent reward-hack review of every published cell; scores from flagged sessions render dimmed — they don't count toward the charts.
Board summary bars are each score relative to the best published model on that board (1.00 = board leader); the printed number is the bench-native score.
Methodology. Rank per bench: valid passes (audited-clean correct cells / problems) desc, Cells with no audit annotation are shown but never scored. then mean normalized performance over the FULL problem deck (cell score / board best per problem; fail/invalid/missing cells count as 0) desc. Hack badge = flagged audited sessions / total audited sessions for that model; flagged = annotation verdict reward_hack | contamination | rubric_leak | suspect, or megakernel_authentic false (mega). Verdicts come from per-run audit YAMLs, not static lint. Hack rate is displayed, never a sort key. Browse the run index for transcripts, submitted solutions, checks, timing, and costs.
RTX PRO 6000· canonical board
all audited agy attempts
Every attempted cell stays visible, including correctness failures, hardware mismatches, contaminated runs, and audit rejects. Only publishable results contribute to the board above.
Gemini 4 Neon (XHigh) wrote a single-launch cooperative SM120 CUDA megakernel (188 CTAs x 512 threads) that scores 36.655x on Kimi-Linear decode (0.157 / 0.166 / 0.176 ms/tok at 2k / 8k / 16k, geomean 6,010 tok/s), taking #1 overall ahead of Claude Opus 5.5 (35.34x). It unpacks centered [-8..+7] int4 weights directly into native SM120 bf16x2 FMA2 (lop3 + sub.rn.bf16x2 + fma.rn.bf16x2) with the zero point folded once per 32-element group, uses PTX release/acquire router barriers with redux.sync top-8 selection, runs FlashMLA on bf16 mma.sync tensor cores with cp.async KV pipelining and adaptive Blackwell persisting-L2 policies, and copies step-0 KV caches inside the single cooperative launch. Verdict is interesting for intra-group bf16x2 accumulation, an unguarded *g_norm_sq reset before grid.sync(), and load_state_dict-only weight repacking.
audited runRTX PRO 6000· canonical board
all audited agy attempts
Every attempted cell stays visible, including correctness failures, hardware mismatches, contaminated runs, and audit rejects. Only publishable results contribute to the board above.
Gemini 4 Neon (XHigh) wrote the GLM-5.2 MoE layer in hand-written SM120 CUDA C++ with inline PTX (cp.async.cg, ldmatrix.x4, bf16 mma.sync m16n8k16, red.relaxed.gpu.global.add.v4.f32) and a 2-kernel streaming GEMV path for small batches (T <= 4). Notably, 0.1010 is its uniterated V1 first draft: after passing an initial GPU smoke test, a sibling cell's deadlock held the shared cuda GPU lock for 2h42m, so this session exited cleanly at 5:29 pm without ever timing or optimizing on GPU.
audited runGemini 4 Neon (XHigh) built a hand-written SM120 CUDA/PTX sparse attention pipeline with exact per-token NSA selection (fp32 block key means split into three bf16 planes for exact tensor-core scoring, causal diagonal mean, register top-8 heaps with exact reference tie-breaking to the larger block index) and a lock-free inverted-index key-stationary attention kernel that stores each (row, block) partial output as saturated signed int8 (pack_i8x4 via cvt.rni.sat.s8.f32) with a compact 8-byte (m, l, scale) header before a softmax-weighted merge. Small D=64 shapes (S <= 1024) take a fused row-stationary kernel with no int8 intermediate.
audited runGemini 4 Neon (XHigh) built a full-context SM120 MegaQwen decode engine combining CUDA Graphs with Programmatic Dependent Launch (griddepcontrol.wait / griddepcontrol.launch_dependents) across QKV, split-KV causal GQA, O-projection, SwiGLU MLP, and Down-projection kernels. Per-step PyTorch CPU MT19937 + Box-Muller decode noise is reproduced identically on GPU inside the timed CUDA Graph, and the entire KV cache stays in exact bf16 with zero quantization or eviction.
audited runSolo rerun of Gemini 4 Neon (XHigh) on GLM-5.3 Fused MoE (following the lock-blocked V1 0.1010 run). Wrote a self-contained 1,977-line SM120 CUDA C++/PTX/TMA kernel suite with zero CPU-GPU synchronization across four adaptive regimes: split-warp 128-thread streaming GEMV for decode (T <= 4); single-barrier hybrid cp.async + 3D-TMA (L2::evict_first when nm==1) with high-priority direct-store shared-expert overlap for small batches; disjoint 2-bucket [BM=128, BM=192] hybrid 3D-TMA with low-priority shared overlap for medium batches; and sequential direct-store shared + universal 3-bucket [BM=128, BM=96, BM=192] schedule minimizing (padded_rows, m_tiles) for large batches, all reducing via SM120 vectorized red.relaxed.gpu.global.add.v4.f32. Lands 0.1105 (#2 overall, +9.4% over V1, fastest on board at T=1, T=512, and T=1000).
audited run