Benchmark · Hard
The per-op kernel deck
Six CUDA/Triton problems, one unlimited agent session per cell, graded against each GPU's own roofline — comparable within a GPU, not across GPUs. Correctness groups the board; flagged audits never reorder it.
methodology + notes
Ordering. Models group by valid passes (audited-clean correct cells), then order by mean peak fraction. The flagged count lists audited sessions that failed the reward-hack review; it never changes the order. Click a model for its per-problem cells, audit chips, and integrity record. Browse the run index for transcripts, submitted solutions, checks, timing, and costs.
RTX PRO 6000. Frontier coding agents on one unlimited autonomous session per problem: Opus 4.8, Fable 5, Grok 4.5, GPT-5.6 Sol, GLM-5.2, MiniMax-M3, DeepSeek V4 Pro, LongCat 2.0, and more. Roofline-graded; every published cell is contamination-checked and reward-hack audited.
H100 PCIe. Opus 4.8, Fable 5, GLM-5.2, MiniMax-M3, DeepSeek V4 Pro, and LongCat 2.0 on a single H100 PCIe with the same containerized harness and roofline grading as the Blackwell deck; peak fraction is measured against H100 dense peaks.
B200. The same models on a single NVIDIA B200 (SM100 Blackwell, HBM3e) with the identical containerized harness and roofline grading; peak fraction is measured against B200 dense peaks (fp8 4500, bf16 2250 TFLOPS), so the same kernel reads as a smaller fraction of a much higher ceiling.
