KernelBench hard · H100

Paged Attention Kimi K3 (256k)

manually audited: clean

Genuine hand-written flash-decoding paged-attention CUDA kernel for H100 (sm_90a via load_inline). One 128-thread block per (split, batch*kv_head) pair gathers live KV pages through block_table with cp.async double/quad- buffered XOR-swizzled smem staging (L2 evict_first policy), computes QK^T and PV via mma.sync.m16n8k16 bf16 with the G GQA query heads packed into the mma M rows, fp32 online softmax in log2 domain, and a flash-decoding split-K whose last-arriving block merges partials via self-zeroing atomic counters (single launch per forward). seq_lens is read live on device per call for tail masking; a generic fallback kernel covers unspecialized shapes. No caching of any kind: output is at::empty_like per call, the kernel launches unconditionally on every forward, no CUDA graphs, no data_ptr keying. No forbidden ops (no SDPA, no vllm/flashinfer). check.py PASS with numeric stress enabled; 0.4958 recomputes exactly from the five logged per-shape fractions.

20260715_204006_kinetic-claude_kinetic-0715_03_paged_attention