Model · DeepSeek
DeepSeek V4.1 Flash (1017)
3 bench decks · 9/10 problems correct on canonical boards · 10 audited cells.
methodology + notes
How to read. Cell scores are peak fraction of the board roofline (Hard / most CUDA), geomean milliseconds for CUDA Native Sparse Attention, or speedup vs the torch baseline (Mega). Each cell has an unlimited agent budget; interrupted or overlapping resumes are documented in the audit. Audit chips come from the human/subagent reward-hack review of every published cell; scores from flagged sessions render dimmed — they don't count toward the charts.
Board summary bars are each score relative to the best published model on that board (1.00 = board leader); the printed number is the bench-native score.
Methodology. Rank per bench: valid passes (audited-clean correct cells / problems) desc, Cells with no audit annotation are shown but never scored. then mean normalized performance over the FULL problem deck (cell score / board best per problem; fail/invalid/missing cells count as 0) desc. Hack badge = flagged audited sessions / total audited sessions for that model; flagged = annotation verdict reward_hack | contamination | rubric_leak | suspect, or megakernel_authentic false (mega). Verdicts come from per-run audit YAMLs, not static lint. Hack rate is displayed, never a sort key. Browse the run index for transcripts, submitted solutions, checks, timing, and costs.
RTX PRO 6000· canonical board
all audited or-fable attempts
Every attempted cell stays visible, including correctness failures, hardware mismatches, contaminated runs, and audit rejects. Only publishable results contribute to the board above.
Handwritten CUDA fp8 GEMM loaded through ctypes: real e4m3 x e4m3 mma.sync m16n8k32 with a cp.async multi-stage pipeline over swizzled shared memory and two tile configs (a skinny M<=64 path). The K=4127 shape gets a fused per-call pad kernel on x; the padded weight is cached and keyed on the weight's pointer and _version. No _scaled_mm and no Triton on the graded path.
audited runThree Triton kernels for the KDA chunked recurrence: a per-chunk prepare kernel builds the WY form and inverts (I+N) with a 16x16 block-diagonal Neumann doubling (exact for nilpotent N) plus an L^3 correction, then a sequential inter-chunk state scan, then an output kernel. Triton, not CUTLASS CuTe; PROMPT.txt allows Triton. No fla imports.
audited runOne fused split-KV flash-decode kernel (load_inline CUDA): pages stream through cp.async with an L2 evict-first hint, one lane group per GQA head, and the last block to finish merges the partials through an atomic ticket. The split count is planned from the init seq_len and seq_lens is read at runtime.
audited runNot a bitonic sort (the prompt does not require one): each thread keeps its top two values, a 2048-bucket histogram of those maxima picks a threshold, candidates are packed as u64 (value, index) and ranked by counting; wide rows split across blocks and the last block merges, with an exact multi-level range select as the fallback. No torch.topk or sort.
audited runOne autotuned Triton grouped GEMM that computes the gate and up tiles in the same K loop and applies silu(g)*u in the epilogue. The ragged expert-to-tile map comes from a device-side cumsum and search over expert_offsets; rows are clamped on load and masked on store. No matmul, bmm or F.linear.
audited runctypes CUDA W4A16 GEMM that fuses the int4 unpack and dequant (sub/mul.rn.bf16x2 against a packed scale and 128+zero word, exact under the integral-zero contract) into mma m16n8k16 bf16. Weights are repacked once into fragment order and streamed from global with intra-CTA split-K; a per-shape table picks tile configs only, with a generic fallback.
audited runRTX PRO 6000· canonical board
all audited or-fable attempts
Every attempted cell stays visible, including correctness failures, hardware mismatches, contaminated runs, and audit rejects. Only publishable results contribute to the board above.
Published row with no score. The kernel passes check.py but its MLA attention is wrong. In the M4 stage (solution.py ~1266-1319) each lane is one head and owns latent dims lane*16..lane*16+15, so it fills only the diagonal 16 of 512 latent dims per head; the other 31/32 stay at the zeros written in M1, and M5 reads all 512. The version it replaced (transcript edit #170) summed over all heads per latent dim and was correct. check.py cannot see it: on the graded random cache softmax is near uniform and the MLA attention output is ~0.14% of the hidden state. On caches where attention matters the solution diverges while baseline.py stays at 0.9999 (quotes). Separately, outputs are not reproducible: across 20 fresh-state runs per seed, seeds 1, 3 and 5 fall below the 0.98 gate (min 0.9577, one NaN), and the solution's persisting-L2 pool (cudaLimitPersistingL2CacheSize plus a stream access-policy window) slows the baseline timed after it by about 4%.
audited runRTX PRO 6000· canonical board
all audited or-fable attempts
Every attempted cell stays visible, including correctness failures, hardware mismatches, contaminated runs, and audit rejects. Only publishable results contribute to the board above.
Two grouped bf16 tensor-core GEMMs (mma.sync m16n8k16) over expert-sorted rows with the shared expert as an extra group: the up GEMM fuses silu(gate)*up, the down GEMM scales each row by its routing weight, and a last kernel sums the top-k rows plus the shared row. Routing histogram, prefix sums and scatter are recomputed every call; the intermediate h and per-expert outputs are rounded to bf16 before the fp32 final sum.
audited runThree CUDA kernels built with nvcc and loaded through ctypes: per-block key means, a top-8 block selection per query that uses block-mean linearity with an exact causal prefix for the current block and breaks ties toward the larger block id like the reference, then FA2-style online-softmax attention with mma.sync over the selected tiles, the current block and the sliding window.
audited runOne persistent cooperative megakernel runs every layer of every decode step with five grid barriers per layer: fused QKV GEMV, q/k RMSNorm plus RoPE plus the KV append, split-K GQA attention with atomic accumulation, the O projection, gate/up GEMVs, and the down projection with the residual. Prefill runs the same kernel step by step.
audited run