Model · DeepSeek

DeepSeek V4.1 Flash (1017)

3 bench decks · 9/10 problems correct on canonical boards · 10 audited cells.

methodology + notes

How to read. Cell scores are peak fraction of the board roofline (Hard / most CUDA), geomean milliseconds for CUDA Native Sparse Attention, or speedup vs the torch baseline (Mega). Each cell has an unlimited agent budget; interrupted or overlapping resumes are documented in the audit. Audit chips come from the human/subagent reward-hack review of every published cell; scores from flagged sessions render dimmed — they don't count toward the charts.

Board summary bars are each score relative to the best published model on that board (1.00 = board leader); the printed number is the bench-native score.

Methodology. Rank per bench: valid passes (audited-clean correct cells / problems) desc, Cells with no audit annotation are shown but never scored. then mean normalized performance over the FULL problem deck (cell score / board best per problem; fail/invalid/missing cells count as 0) desc. Hack badge = flagged audited sessions / total audited sessions for that model; flagged = annotation verdict reward_hack | contamination | rubric_leak | suspect, or megakernel_authentic false (mega). Verdicts come from per-run audit YAMLs, not static lint. Hack rate is displayed, never a sort key. Browse the run index for transcripts, submitted solutions, checks, timing, and costs.

Board summarybars = share of each board's best model · numbers = bench-native score
Hard
21.2%6/6
Mega
—0/1
CUDA
18.3%3/3
Hardor-fable
6/6 pass6 audited

RTX PRO 6000· canonical board

FP8 GEMMpass
32.4%clean
session 2h 7m
KDA CUTLASSpass
4.0%clean
session 2h 1m
Paged Attentionpass
45.9%clean
session 49m
TopK Bitonicpass
9.0%interesting
session 1h 31m
Sonic MoEpass
10.6%clean
session 1h 22m
W4A16 GEMMpass
25.1%interesting
session 3h 1m

all audited or-fable attempts

Every attempted cell stays visible, including correctness failures, hardware mismatches, contaminated runs, and audit rejects. Only publishable results contribute to the board above.

RTX_PRO_6000 · FP8 GEMMclean
32.42%publishable

Handwritten CUDA fp8 GEMM loaded through ctypes: real e4m3 x e4m3 mma.sync m16n8k32 with a cp.async multi-stage pipeline over swizzled shared memory and two tile configs (a skinny M<=64 path). The K=4127 shape gets a fused per-call pad kernel on x; the padded weight is cached and keyed on the weight's pointer and _version. No _scaled_mm and no Triton on the graded path.

audited run
RTX_PRO_6000 · KDA CUTLASSclean
4.03%publishable

Three Triton kernels for the KDA chunked recurrence: a per-chunk prepare kernel builds the WY form and inverts (I+N) with a 16x16 block-diagonal Neumann doubling (exact for nilpotent N) plus an L^3 correction, then a sequential inter-chunk state scan, then an output kernel. Triton, not CUTLASS CuTe; PROMPT.txt allows Triton. No fla imports.

audited run
RTX_PRO_6000 · Paged Attentionclean
45.86%publishable

One fused split-KV flash-decode kernel (load_inline CUDA): pages stream through cp.async with an L2 evict-first hint, one lane group per GQA head, and the last block to finish merges the partials through an atomic ticket. The split count is planned from the init seq_len and seq_lens is read at runtime.

audited run
RTX_PRO_6000 · TopK Bitonicinteresting
8.97%publishable

Not a bitonic sort (the prompt does not require one): each thread keeps its top two values, a 2048-bucket histogram of those maxima picks a threshold, candidates are packed as u64 (value, index) and ranked by counting; wide rows split across blocks and the last block merges, with an exact multi-level range select as the fallback. No torch.topk or sort.

audited run
RTX_PRO_6000 · Sonic MoEclean
10.60%publishable

One autotuned Triton grouped GEMM that computes the gate and up tiles in the same K loop and applies silu(g)*u in the epilogue. The ragged expert-to-tile map comes from a device-side cumsum and search over expert_offsets; rows are clamped on load and masked on store. No matmul, bmm or F.linear.

audited run
RTX_PRO_6000 · W4A16 GEMMinteresting
25.13%publishable

ctypes CUDA W4A16 GEMM that fuses the int4 unpack and dequant (sub/mul.rn.bf16x2 against a packed scale and 128+zero word, exact under the integral-zero contract) into mma m16n8k16 bf16. Weights are repacked once into fragment order and streamed from global with intra-CTA split-K; a per-shape table picks tile configs only, with a generic fallback.

audited run
Megaor-fable
0/1 pass1 audited

RTX PRO 6000· canonical board

Kimi-Linear Decodewrong
bug
no resultwrong
ptx

all audited or-fable attempts

Every attempted cell stays visible, including correctness failures, hardware mismatches, contaminated runs, and audit rejects. Only publishable results contribute to the board above.

RTX PRO 6000 Blackwell · Kimi-Linear Decodebug
no scorecorrectness failed

Published row with no score. The kernel passes check.py but its MLA attention is wrong. In the M4 stage (solution.py ~1266-1319) each lane is one head and owns latent dims lane*16..lane*16+15, so it fills only the diagonal 16 of 512 latent dims per head; the other 31/32 stay at the zeros written in M1, and M5 reads all 512. The version it replaced (transcript edit #170) summed over all heads per latent dim and was correct. check.py cannot see it: on the graded random cache softmax is near uniform and the MLA attention output is ~0.14% of the hidden state. On caches where attention matters the solution diverges while baseline.py stays at 0.9999 (quotes). Separately, outputs are not reproducible: across 20 fresh-state runs per seed, seeds 1, 3 and 5 fall below the 0.98 gate (min 0.9577, one NaN), and the solution's persisting-L2 pool (cudaLimitPersistingL2CacheSize plus a stream access-policy window) slows the baseline timed after it by about 4%.

audited run
CUDAor-fable
3/3 pass3 audited

RTX PRO 6000· canonical board

GLM-5.3 Fused MoEpass
9.7%clean
session 5h 8m
DeepSeek NSApass
0.266 msclean
session 5h 2m
MegaQwen Decodepass
8.1%interesting
session 4h 13m

all audited or-fable attempts

Every attempted cell stays visible, including correctness failures, hardware mismatches, contaminated runs, and audit rejects. Only publishable results contribute to the board above.

RTX_PRO_6000 · GLM-5.3 Fused MoEclean
9.65%publishable

Two grouped bf16 tensor-core GEMMs (mma.sync m16n8k16) over expert-sorted rows with the shared expert as an extra group: the up GEMM fuses silu(gate)*up, the down GEMM scales each row by its routing weight, and a last kernel sums the top-k rows plus the shared row. Routing histogram, prefix sums and scatter are recomputed every call; the intermediate h and per-expert outputs are rounded to bf16 before the fp32 final sum.

audited run
RTX_PRO_6000 · DeepSeek NSAclean
0.266 mspublishable

Three CUDA kernels built with nvcc and loaded through ctypes: per-block key means, a top-8 block selection per query that uses block-mean linearity with an exact causal prefix for the current block and breaks ties toward the larger block id like the reference, then FA2-style online-softmax attention with mma.sync over the selected tiles, the current block and the sliding window.

audited run
RTX_PRO_6000 · MegaQwen Decodeinteresting
8.12%publishable

One persistent cooperative megakernel runs every layer of every decode step with five grid barriers per layer: fused QKV GEMV, q/k RMSNorm plus RoPE plus the KV append, split-K GQA attention with atomic accumulation, the O projection, gate/up GEMVs, and the down projection with the residual. Prefill runs the same kernel step by step.

audited run