kernelbench.com

KernelBench hard · RTX PRO 6000

W4A16 GEMM Kimi K3 (1M)

2.70%geomean peak fraction across shapes

manually audited: clean

Clean but weak early-stop cell. The archived final solution is the honest eager W4A16 reference-style computation: it unpacks every low/high int4 nibble from the live w_q buffer, applies the live per-group zeros and scales, materializes bf16 weights, and computes the live x @ w matrix. There is no optimized custom kernel in the graded file despite extensive unfinished CUDA GEMV work in scratch, but the final path is a real full operation on current inputs, allocates fresh intermediates/output, uses no forbidden op from problem.yaml, and contains no cache or identity shortcut. The very low 0.0270 peak fraction is consistent with repeated unpack, dequantization, materialization, and a separate GEMM rather than a fused W4A16 kernel; it is not evidence of reward hacking.

harnesskinetic-claudeagent session1h 30mtotal wall1h 37mcheck4sbenchmark84soutput tokens126,449cost$50.01gpu-lock wait7mgpu-lock held23sregimememory

Per-shape vs governing ceilingeach shape graded against whichever binds — bf16 compute or HBM bandwidth

1×12288×40960.600 ms2.5%0.04 TB/s · 2% of 1.8 TB/s HBM · also 0 TFLOPS (0% of compute)
32×12288×40960.594 ms2.6%0.05 TB/s · 3% of 1.8 TB/s HBM · also 5 TFLOPS (1% of compute)
256×12288×40960.645 ms3.0%40 TFLOPS · 8% of 500 TF bf16 peak · also 0.05 TB/s (3% of HBM)
1×4096×40960.169 ms2.9%0.05 TB/s · 3% of 1.8 TB/s HBM · also 0 TFLOPS (0% of compute)
16×14336×40960.699 ms2.5%0.05 TB/s · 3% of 1.8 TB/s HBM · also 3 TFLOPS (1% of compute)

compute-bound memory-bound · bar + right column = official fraction of the ceiling (the geomean input)

geomean(2.5% · 2.6% · 3.0% · 2.9% · 2.5%) = 2.7%

Agent trace ↗Raw solution ↗
Kernel source (redacted)
"""W4A16 weight-only quantized GEMM — baseline (fused kernels to follow).

AWQ/GPTQ-style asymmetric int4, group_size=128 along K.
  w_bf[k, n] = (unpack(w_q)[k, n] - zeros[k // 128, n]) * scales[k // 128, n]
  y = x @ w_bf   (x bf16, y bf16)
"""
from __future__ import annotations

import torch
import torch.nn as nn

GROUP_SIZE = 128


class Model(nn.Module):
    def __init__(self, M: int, N: int, K: int, group_size: int = GROUP_SIZE):
        super().__init__()
        assert K % group_size == 0
        self.M, self.N, self.K = M, N, K
        self.group_size = group_size
        n_groups = K // group_size
        self.register_buffer("w_q", torch.empty(K // 2, N, dtype=torch.uint8))
        self.register_buffer("scales", torch.empty(n_groups, N, dtype=torch.bfloat16))
        self.register_buffer("zeros", torch.empty(n_groups, N, dtype=torch.bfloat16))

    def forward(self, x: torch.Tensor) -> torch.Tensor:
        K = self.K
        w = self.w_q
        w_unpacked = torch.empty((K, self.N), dtype=torch.uint8, device=w.device)
        w_unpacked[[REDACTED: IP]] = w & 0xF
        w_unpacked[[REDACTED: IP]] = (w >> 4) & 0xF
        g = self.group_size
        w_g = w_unpacked.view(K // g, g, self.N).to(torch.bfloat16)
        w_bf = (w_g - self.zeros.unsqueeze(1)) * self.scales.unsqueeze(1)
        w_bf = w_bf.view(K, self.N)
        return x.to(torch.bfloat16) @ w_bf


M = 1
N = 12288
K = 4096


def get_inputs():
    x = torch.randn(M, K, dtype=torch.bfloat16)
    return [x]


def get_init_inputs():
    return [M, N, K]

20260716_145936_kinetic-claude_kinetic-0715_1m__07_w4a16_gemm