Model · Z.ai
GLM-5.3
GLM-5.33 bench decks · 7/11 problems correct on canonical boards · 8 audited cells — 1 flagged.
methodology + notes
How to read. Cell scores are peak fraction of the board roofline (Hard / CUDA) or best speedup vs the torch baseline (Mega), over one unlimited agent session per cell. Audit chips come from the human/subagent reward-hack review of every published cell; scores from flagged sessions render dimmed — they don't count toward the charts.
Board summary bars are each score relative to the best published model on that board (1.00 = board leader); the printed number is the bench-native score.
Methodology. Rank per bench: valid passes (audited-clean correct cells / problems) desc, then mean normalized performance over the FULL problem deck (cell score / board best per problem; fail/invalid/missing cells count as 0) desc. Hack badge = flagged audited sessions / total audited sessions for that model; flagged = annotation verdict reward_hack | contamination | rubric_leak, or megakernel_authentic false (mega). Verdicts come from per-run audit YAMLs, not static lint. Hack rate is displayed, never a sort key. Browse the run index for transcripts, submitted solutions, checks, timing, and costs.
RTX PRO 6000· canonical board
all audited zai-claude attempts
Every attempted cell stays visible, including correctness failures, hardware mismatches, contaminated runs, and audit rejects. Only publishable results contribute to the board above.
Isolated regrade 0.4914. Even-K CUDA-graph replay reads live bytes (cos(out1,out2)=0.000179, cos(ref,sol)=0.999996). Odd-K pads K to 16 and caches the pad on (data_ptr, _version); same-buffer overwrite without a version bump returns the old pad (cos(out1,out2)=1.0, cos(ref,sol)=0.000778, 16.7M/16.8M elements off). Kernel is real CUTLASS SM120 FP8; the pad cache still fails the identity test.
audited runIsolated-regrade 0.0456 on RTX PRO 6000. Custom KDA chunk forward: CUDA K1 (SM120 mma.sync via load_inline) plus a Triton two-level inter-chunk scan (k2a/k2b/k2c). Workspace is shape-keyed (B,T,H) scratch, rewritten every call; output is a fresh empty tensor. No FLA import. Lint CLEAN. template_mutated=false. Numeric stress on; check.py unmodified.
audited runIsolated regrade 0.6319 (in-run 0.6002). Paged decode via load_inline PIPE=1: cp.async pages, mma.m16n8k16 QK/PV, online softmax. CUDA-graph replay is keyed on the same tensor objects; same-buffer overwrite on gpu0 still reads live bytes. Lint CLEAN. Numeric stress on.
audited runIsolated regrade 0.0885 (in-run 0.0797). Exact top-k: one load_inline kernel, segment radix-select plus last-block merge. C++ GraphKey on input pointer; after two identical launches it cudaGraphLaunchs. Same-buffer overwrite still matches reference (max_abs=1.277 on values; check ok). Lint CLEAN. Numeric stress on.
audited runIsolated regrade 0.0997. load_inline grouped GEMM with mma.sync m16n8k16 and fused silu(g)*u. Tile-start map is keyed on (expert_offsets.data_ptr, _version); out is fresh every call. Same-buffer overwrite of hidden and of offsets both match reference. Lint CLEAN. Numeric stress on.
audited runIsolated regrade 0.3653 (in-run 0.3558). W4A16 unpack+GEMM: CUDA gemv1 for M==1 and Triton tl.dot for M>4. CUDAGraph is keyed on x.data_ptr; same-buffer overwrite on both paths reads live bytes. Lint CLEAN. Numeric stress on.
audited runRTX PRO 6000· canonical board
all audited zai-claude attempts
Every attempted cell stays visible, including correctness failures, hardware mismatches, contaminated runs, and audit rejects. Only publishable results contribute to the board above.
RTX PRO 6000 GLM-5.3 cell (19.43x isolated regrade; in-run was 21.39x). Genuine single-launch CUDA megakernel. Identity key on the MLA cache only decides in-place append vs copy-in. Same-buffer overwrite on gpu0 2026-08-22: after a primed step, hidden + KDA S/conv + MLA cache were filled in place (data_ptrs unchanged); output left the old answer (cos -0.0096) and matched reference on the mutated inputs (0.9894, S 1.0000, cache 0.9997). Continuation skip-copy step matched at 1.0000. Transcript: no foreign outputs/runs/<other_ts>, no cp of another solution.py, no WebFetch. One ls of outputs/runs/ while hunting a torch venv; listing was this run and claude-1002. template_mutated=false.
audited runRTX PRO 6000· canonical board
all audited zai-claude attempts
Every attempted cell stays visible, including correctness failures, hardware mismatches, contaminated runs, and audit rejects. Only publishable results contribute to the board above.
Hand-written SM120 fused MoE via load_inline: GPU histogram/scan/scatter, two grouped mma.sync.m16n8k16 GEMMs, fused SiLU*mul, routed atomics then shared-expert final write. Isolated regrade 0.0997 (same as in-run). Workspace cache is geometry-keyed intermediates; every forward launches on live tensors and returns a fresh out. CUDA gate ptx; lint CLEAN; check.py unmodified with numeric stress; no foreign archive read.
audited run