Model · Google
Gemini 4 Carbon (High)
2 bench decks · 4/4 problems correct on canonical boards · 4 audited cells.
methodology + notes
How to read. Cell scores are peak fraction of the board roofline (Hard / most CUDA), geomean milliseconds for CUDA Native Sparse Attention, or speedup vs the torch baseline (Mega). Each cell has an unlimited agent budget; interrupted or overlapping resumes are documented in the audit. Audit chips come from the human/subagent reward-hack review of every published cell; scores from flagged sessions render dimmed — they don't count toward the charts.
Board summary bars are each score relative to the best published model on that board (1.00 = board leader); the printed number is the bench-native score.
Methodology. Rank per bench: valid passes (audited-clean correct cells / problems) desc, Cells with no audit annotation are shown but never scored. then mean normalized performance over the FULL problem deck (cell score / board best per problem; fail/invalid/missing cells count as 0) desc. Hack badge = flagged audited sessions / total audited sessions for that model; flagged = annotation verdict reward_hack | contamination | rubric_leak | suspect, or megakernel_authentic false (mega). Verdicts come from per-run audit YAMLs, not static lint. Hack rate is displayed, never a sort key. Browse the run index for transcripts, submitted solutions, checks, timing, and costs.
RTX PRO 6000· canonical board
all audited agy attempts
Every attempted cell stays visible, including correctness failures, hardware mismatches, contaminated runs, and audit rejects. Only publishable results contribute to the board above.
Re-graded 2026-11-16 under grader v3 on tetra: 24.68x (23.76x before; see regrade_v3). Original audit: Gemini 4 Carbon wrote one persistent CUDA kernel, 376 CTAs (two per SM) held together by a single-word release/acquire grid barrier, that runs the whole decode token: three KDA layers, the absorbed MLA layer and a 64-expert top-8 plus shared-expert MoE after each. The int4 GEMVs build each weight as a bf16 "128 + nibble" with one OR (0x43004300) and fold the zero point in once per group against precomputed 32-element activation sums; activations stay fp32 from the bf16 RMSNorm output, so it is a true W4A16 GEMV. MLA attention runs in the 512-dim latent space on bf16 mma.sync tensor cores with a two-stage cp.async pipeline over up to 128 KV splits, and dense weights are pinned in L2 with evict_last hints while routed experts stream evict_first.
audited runRTX PRO 6000· canonical board
all audited agy attempts
Every attempted cell stays visible, including correctness failures, hardware mismatches, contaminated runs, and audit rejects. Only publishable results contribute to the board above.
Gemini 4 Carbon wrote the GLM-5.2 MoE layer as hand-written CUDA with inline PTX for SM120. One 512-thread CTA counting-sorts the (token, slot) pairs by expert and builds a tile table (tile height 32, 64 or 128 rows picked from the average tokens per expert); two grouped GEMMs then run on TMA-loaded weights with an mbarrier pipeline, cp.async-gathered activation rows, ldmatrix and bf16 mma.sync, with silu(gate)*up fused into the first GEMM's epilogue and the routing weight applied in the second before red.global.add.v4.f32 scatters rows into an fp32 buffer. The shared expert is a dense TMA GEMM pair that writes that buffer first, a cast kernel produces bf16, and T <= 2 takes a separate two-kernel persistent GEMV path.
audited runGemini 4 Carbon wrote NSA block selection and sparse attention as four raw CUDA launches with inline-PTX bf16 mma.sync. A key-mean kernel stores fp32 block means of K; a per-query-block kernel scores every earlier block as q·mean(k) (exact fp32 FMA for D=64; for D=128 a bf16 tensor-core prefilter to 16 candidates, then fp32 rescoring wherever a Cauchy-Schwarz error bound cannot decide membership), takes the causal mean of the diagonal block from its tensor-core scores, keeps a top-8 with ties to the larger block index, attends the diagonal block and the previous block (whole if selected, else only the sliding-window tail), and scatters each remote selection into a per-key-block bucket. A persistent key-stationary kernel loads each selected K/V block once and attends all of its bucketed queries, and an epilogue merges the bf16 partials by log-sum-exp.
audited runGemini 4 Carbon wrote one cooperative CUDA megakernel (188 blocks x 512 threads, five grid.sync per layer) that runs every decode step of all four layers in a single launch: fused RMSNorm and QKV GEMV, q/k-norm plus RoPE plus the KV append, split-KV flash decoding with a last-arriver merge, O-proj, SwiGLU and down-proj with atomic residual adds. KV stays bf16 and the softmax keeps its running max. Its one unusual move: the timed decode's random inputs are drawn on the CPU while the kernel is already running and streamed to the GPU through mapped host memory.
audited run