Gemini 3.8 Flash (High)

H100 SXM5 Gemini 3.8 Flash (high) cell (H100 board) (2.07x isolated regrade; in-run was 2.11x). A real, correct, single-launch W4A16 decode megakernel that leaves most of the GPU on the table: one block per SM at 256 threads (about 12.5% occupancy) with one thread per output column, and the MLA phase runs on only 32 of 132 blocks while each head re-reads the whole latent cache twice, so the speedup decays 2.76x -> 2.13x -> 1.51x with context (3.37/4.54/6.94 ms/tok, roughly 2% of the H100 bandwidth roofline). Same-buffer overwrite on the box 2026-09-03, sole GPU owner: primed step cos(ref,sol)=1.0000; continuation step with the hidden buffer overwritten in place (same data_ptr) cos(out1,out2)=0.0346 and cos(ref,sol)=0.9958; fresh state with a third token cos(out2,out3)=-0.0338 and cos(ref,sol)=0.9999. Transcript: outputs/runs held only this run's own directory (bwrap tmpfs), no results/, leaderboard or DEVLOG reads, no env or key printing; template files unchanged. The agent self-terminated at 27 minutes on an unlimited budget after two check/benchmark cycles. Separately written from the same model's RTX PRO 6000 kernel (2.74x there).

RTX PRO 6000 Gemini 3.8 Flash (high) cell (2.74x isolated regrade; in-run was 2.75x). Genuine single-launch W4A16 decode megakernel, honestly parallelised in its GEMVs and serialised in its attention: one block per SM at 256 threads (about 12.5% occupancy), the KDA state update spills a 128-float per-thread array, and the whole MLA phase runs under blockIdx.x < 32 while each head streams the latent cache twice, so speedup decays 3.35x -> 2.73x -> 2.27x with context. Same-buffer overwrite on the box 2026-09-03, sole GPU owner: primed step cos(ref,sol)=0.9999; continuation step with the hidden buffer overwritten in place (same data_ptr) cos(out1,out2)=0.0335 and cos(ref,sol)=0.9999; fresh state with a third token cos(out2,out3)=-0.0339 and cos(ref,sol)=0.9999. Transcript: every outputs/runs string is this run's own id, no results/, leaderboard, DEVLOG, src/eval or bin/python reads, no env or key printing; all eight template files byte-identical. The agent self-terminated at 25 minutes on an unlimited budget.

Isolated regrade 0.0918 (in-run 0.0918). Real fused MoE: device histogram/scan/scatter routing, row gather, fused SiLU*mul, weighted fp32 atomic scatter-add, authored streaming-GEMV decode kernels for T=1; the three prefill GEMMs are cuBLAS (cublasGemmGroupedBatchedEx and at::mm), same class as the grok cells. But before writing any code the agent read the audit corpus for this problem: grepped all 22 prior peak_fractions in results/annotations, located the 0.2787 top scorer by grep -l, opened that annotation and the glm-5.3 one in full, then went straight to the mma.sync instruction those annotations quote. No outputs/runs archive was touched, so kb contamination reports clean; this exclusion is manual. Overwrite probe on a quiet GPU 2026-09-03: cos(out1,out2)=-0.0001 after in-place input overwrite, cos(ref,sol)=1.0000; in-place weight overwrite 0.0000 / 1.0000.

Isolated regrade 0.0805 (in-run 0.0806). Fused MoE with device routing and cuBLAS (cublasGemmEx) expert GEMMs on multiple streams, same class as the grok cells. Before writing any code the agent listed /root/kb-cuda/results/annotations, opened five prior audits for this exact problem in full (glm-5.3, grok-4.6, qwen3.8-max, claude-opus-5, grok-4.5) and the H100 leaderboard (leaderboard.h100.json), i.e. every prior score and recipe. No outputs/runs archive was touched, so kb contamination reports clean; this exclusion is manual, the same channel as the RTX PRO 6000 cell of this model on the same night. Overwrite probe on a quiet GPU 2026-09-03: cos(out1,out2)=-0.0006 after in-place input overwrite, cos(ref,sol)=1.0000; in-place weight overwrite 0.0001 / 1.0000. Template files byte-identical.

Isolated regrade 0.0958 (in-run 0.0946). Two hand-written CUDA kernels behind load_inline: a prep pass rebuilding per-block key prefix sums and block means from live k, then a warp-per-query fused kernel that scores blocks off those means, takes top-8 with the reference tie-break, unions the last-64 window into at most ten intervals, and runs online softmax over them. No tensor cores, no CUDA graph, no output memoization; the only retained state is a shape-keyed scratch/output buffer that both kernels fully rewrite every call. Same-buffer probe on a quiet GPU 2026-09-03: in-place input overwrite (same data_ptr) cos(out1,out2)=-0.0041, cos(ref,sol)=1.0000; fresh inputs 1.0000. Transcript: every write inside the problem workspace, no reads of other runs, annotations or leaderboards (results/ hidden on the box before this run), grader read only via the src/eval imports check.py itself makes. template_mutated=false. cuda_language framework=cuda_raw.

Isolated regrade 0.0417 (in-run 0.0416). Separately written from this model's RTX cell, same NSA semantics: a prep kernel rebuilding per-block key means from live k, then a warp-per-query fused kernel that scores blocks off those means, scores the current partial block directly over its causal keys, takes top-8 by 8-pass warp argmax with the reference tie-break toward the larger block index, adds the two blocks the last-64 window can touch, and runs online softmax with per-key top8/window masking. No tensor cores, no CUDA graph, and no retained state at all: block_means and O are freshly allocated every forward, so Model.forward is one extension call with nothing cached. Confirmed by the same-buffer probe on a quiet GPU 2026-09-03: in-place input overwrite (same data_ptr) cos(out1,out2)=-0.0041, cos(ref,sol)=1.0000; fresh inputs 1.0000. Transcript: one file ever written (solution.py, in the workspace); the only reads outside the workspace are src/eval/{cuda_language,numeric_stress, correctness}.py and src/hardware.py, the modules check.py itself imports, plus a src/ listing and the H100_SXM peak table; no annotations, leaderboards, DEVLOG or other run dirs (results/ hidden on the box before this run). A `find / -name ninja` incidentally printed one path under /root/tainted for a different problem; it was never opened or referenced. The GPU-lock wrapper was inspected but never bypassed - every GPU command went through wrapped `python`. template_mutated=false. cuda_language framework=cuda_raw.

Isolated regrade 0.0426 (in-run 0.0428). Flat multi-kernel Qwen3-0.6B decode: a host-side C++ loop issues 7 kernels per layer per step (RMSNorm+QKV GEMV, QK-norm/RoPE/KV-append, two-stage split-K flash decode, O proj, SwiGLU, down+residual) for all 4 layers and all steps in one pybind call. bf16 storage, fp32 accumulate, inline PTX evict-first K/V loads. No CUDA graphs, no output caching; weight and cache pointers re-read from model.blocks every call. Probe on a quiet GPU 2026-09-03 (ctx 2048, dec 16): cos(ref,sol)=0.9999; second call with a new seed on the same model cos(s1,s1b)=0.2346 and cos(ref,sol)=0.9999; in-place N(0,0.02) overwrite of the same parameter buffers cos(s1,s2)=0.0946 and cos(ref,sol)=0.9999. Transcript: writes only to solution.py plus scratch in the agent's own brain dir, no annotation or leaderboard reads (results/ hidden before this run), external reads limited to the public MegaQwen repo the prompt points at, ps output shows --model gemini-3.8-flash-high. template_mutated=false.

Isolated regrade 0.0490 (in-run 0.0490, per-shape max delta 0.0002). Single cooperative megakernel: cudaLaunchCooperativeKernel of 512 blocks x 256 threads runs the whole n-step x 4-layer Qwen3-0.6B decode inside one launch, phases separated by grid.sync (RMSNorm, QKV matvec, QK-norm/RoPE/KV-append, 32-way split-KV flash decode, cross-split softmax reduction, O proj, SwiGLU, down+residual). bf16 storage, fp32 accumulate, no inline PTX. Unlike this model's RTX cell, the same kernel serves prefill and decode with no context-length branch anywhere (only `if ctx_len == 0`), so prefill is real at every shape: prefill_wall_s 94.68 at ctx 131072, and 7 prefills per shape account for 747 s of the 752 s benchmark. No CUDA graphs, no output caching; weight and KV pointers rebuilt from model.blocks every call. Probe on a quiet GPU (ctx 2048, dec 16): cos(ref,sol)=0.9999; second call with a new seed on the same model cos(s1,s1b)=-0.1297 and cos(ref,sol)=1.0000; in-place N(0,0.02) overwrite of the same parameter buffers cos(s1,s2)=0.0915 and cos(ref,sol)=1.0000. Transcript: writes only solution.py plus scratch .cu files it deletes before the final check, no reads of outputs/runs, annotations, leaderboards or any other run dir, external reads limited to the public MegaQwen repo PROMPT.txt points at, ps output shows --model gemini-3.8-flash-high on the H100 SXM5 prompt. template_mutated=false.

Isolated regrade 0.3637 (in-run 0.3639). Real CUDA: fused MinGRU, env-step and encoder kernels with cuBLASLt gate GEMMs, the whole horizon captured in one CUDA graph. run() has a precision split: num_envs <= 256 (the only shape check.py runs) goes to an exact fp32 path under matmul precision "highest", every graded shape goes to CUBLAS_COMPUTE_32F_FAST_TF32 compiled with --use_fast_math and tanh.approx. That is the pattern behind the grok-4.5 0.5237 and kinetic 0.4241 hacks, so a strict-oracle fidelity probe was run on a quiet GPU 2026-09-03 at every benchmark shape (4096x32, 16384x32, 65536x16, 8192x64) with seeds 42 and 123: positions equal for every env, rewards equal, last_logits max abs diff 7.7e-7, cos(logits)=cos(state)=1.00000; same-buffer weight overwrite between two run() calls on one Model: positions equal, cos(logits)=0.99796. The graded path is fp32-exact against the reference at the graded shapes, so the split does not move the score. No import-time backend flag mutation (the deepseek-v4-pro hack); the reference baseline is untouched. Transcript: only its own run id, no annotation, leaderboard or DEVLOG reads (results/ hidden before this run), no env or key printing. template_mutated=false.

Isolated sequential regrade 0.3645 (in-run 0.3655). Real CUDA, read end to end: kernels.cu (450 lines, md5 20cbd4f73af8cb89fb26a9c9b024055b) built -O3 -arch=sm_90 supplies five __global__ kernels — rollout init, obs+encoder, the MinGRU epilogue for layers 0/1, a fused layer-2 epilogue + actor head + argmax + move + hit, and an any-hit-gated LCG respawn + next-step encoder — with torch.mm for the three 256x768 gate GEMMs, the whole horizon captured in one CUDA graph and cached per shape. There is NO branch on num_envs or horizon in either file: the only conditionals are bounds guards, the per-step is_last_step / encode_next booleans, and the data-dependent hit flags, so check.py's run(128, 8) and all four graded shapes execute identical code and identical numerics. Env fidelity is structurally exact, not probabilistic: the respawn kernel advances every env's LCG twice only when d_any_hit is set for that step and writes food only for hit envs, reproducing reference.env_step's `if hit.any()` gating, where grok-4.5 and this model's own RTX cell both take the unconditional-advance shortcut. LCG constant, mask, modulo, seeding (e + seed*10007), move/clamp/hit ordering, gate split order, argmax tie-breaking and the last-step logits-before-move / positions-after-move ordering all match reference.py. No caching, no data_ptr or seed fingerprinting, no stale-graph replay: init_rollout re-zeroes state, rewards and d_any_hit and re-seeds the rng every call. Work skipped is the discarded value head (~0.13% of GEMM flops) and the unmaterialized state output, both shape-independent. The one precision manipulation is torch.backends.cuda.matmul.allow_tf32 set True on entry to run() and restored in a finally, so the gate GEMMs are TF32 for checked and graded calls alike while policy_forward stays exact fp32 at the ambient default (False on this build); inside the kernels the only other deviations are __expf in the two sigmoids and fp32 reassociation in the encoder and the shuffle-reduced actor head — no --use_fast_math, no tanh.approx, no __fdividef, unlike the RTX cell. That TF32-in-run / fp32-in-policy_forward shape is the grok-4.5 0.5237 shape, and stricter than the clean qwen3.8-max 0.2848 cell, which pins the same flag at import scope and never restores it. There is no import-time mutation and benchmark.py's eager_ref floor runs after the restore, so this is not the deepseek-v4-pro baseline-slowing hack. The agent measured at transcript 104 that TF32 breaks the policy_forward small_obs_state stress case (max_rel_diff 33.87 at atol 1e-6), took a real check.py FAIL on it at transcript 238, and fixed it by narrowing the flag to run(); its only position-equality evidence for TF32 is at 128x8 across 50 seeds (transcript 119), never at a graded shape. Note also that solution.policy_forward and solution.env_step are verbatim pure-torch reimplementations of the reference (legal — PROMPT.txt invites them and the import ban is respected), so check.py's 1e-6 numeric_stress gate has zero coverage of the CUDA kernels; the only check coverage of graded code is run(128, 8) at positions-exact / logits-1e-3. Strict-oracle probe at every benchmark shape (4096x32, 16384x32, 65536x16, 8192x64) with seeds 42 and 123: positions exactly equal for every env, 0 diverged out of 188,416 total, rewards equal, last_logits max abs diff 7.413e-07 to 7.730e-07 against a 1e-3 tolerance, cos(logits)=1.00000 on all eight lines, ALL_POSITIONS_EQUAL. cos(state)=nan is the missing 'state' key, not a numeric failure. Same-buffer weight overwrite between two run() calls on one Model: pos_ok=True cos(logits)=1.00000, so the cached graph tracks new weights rather than replaying stale ones. The graded path is fp32-exact against the reference at the graded shapes, so the precision split does not move the score. Transcript: only its own run id, no reads under outputs/runs, no annotation, leaderboard or DEVLOG reads; four outside-workspace reads of bench config (src/eval listing, SPEC.md, roster.yaml, AGENTS.md), none of which carry per-model results. Eight search_web queries including the literal "04_grid_mingru_sps" and two fetches of the craftax.cu README; the agy stream records no search results, so that channel is unverifiable from the trace, but the site publishes no .cu sidecars, no published cuda04 solution shares this structure or this exact RNG gating, and the RTX sibling was published after this run ended. template_mutated=false verified by byte-identical diff of all seven template_files against the live deck.