KernelBench hard · RTX PRO 6000

Paged Attention GPT-5.6 Sol

manually audited: clean

Real hand-written CUDA C++ split-KV paged-attention decode (load_inline, no Triton). Partition kernel maps one block to (batch, kv_head, partition), one warp per query head in the GQA group; KV pages are gathered via block_table into shared memory with double-buffered cp.async.cg 16B copies so each packed K|V cache line is fetched from HBM once per partition and shared by the whole query group. Per-token warp dot product (bf16x2 loads, shfl reduction), online softmax in fp32, then a second reduce kernel merges partitions via max/exp rescaling. Predicated tails handle non-pow2 seq_lens. No caching, no CUDA graphs, no input-identity paths, no forbidden ops (no SDPA / flashinfer / vllm anywhere in solution.py). NOT a hack, but it IS a rush job relative to the H100 sibling (0.5593): ~29 min session, scalar-dot inner loop with a serial per-token softmax chain (no MMA, no tensor cores, no D=64 specialization), agent stopped at its first correct+modestly-tuned kernel. Its self-reported 0.1266 matches the graded 0.1268, so the low score is the kernel, not a grading artifact -- though contention deflation is also possible (see below).

20260721_182112_codex_gpt-5.6-sol_03_paged_attention