llama.cpp 发布: b11177

来源摘要

CUDA: fuse RMS_NORM + SCALE into one kernel (#29393) - #28068 builds the GDN q/k l2norm as ggml_scale(ggml_rms_norm(x, eps/n), 1/sqrt(n)). This adds 2 SCALE nodes per GDN layer, 96 extra kernel launches per ubatch on Qwen3.8-27B (48 GDN layers). - The extra kernels take no measurable GPU time, but each launch has a host/driver cost. It is small with plain batch processing and about 10x larger with draft-mtp speculative decoding. - rms_norm_f32 gets a do_scale flag, the same pattern as do_multiply/do_add, so the fused path shares the kernel, the reduction and the launcher. It computes scale * (rsqrt(mean + eps) * x), which matches the unfused rms_norm + scale bit for bit, so #28068 numerics are kept. - Fusion only fires when SCALE has no bias and the rms_norm output has a single consumer (ggml_can_fuse). - Metal (#28948) and SYCL (#28931) already fuse the same pattern. Measured on 2x GTX 1080 Ti (sm_61, PCIe 3.0 x16 + x4), i7-13700KF, Windows 11, driver 582.66, CUDA 12.9. Qwen3.8-27B-UD-Q4_K_XL, -ngl 99 -ts 53,47 -ot token_embd=CPU, master fee39dd92. llama-bench -ub 128,512 -p 512,2048 -n 128 -r 5, tok/s: build pp512@128 pp2048@128 pp2048@512 tg128 master 367.7 419.1 385.4 12.90 master + fix 372.6 420.6 388.6 12.98 +1.3% +0.4% +0.8% +0.6% llama-server cold prefill, -c 56000 -ub 128 -b 2048, draft-mtp n-max 3 p-min 0.5, mean of 2 rounds x 3 reps: build pp 8000 pp 20000 master 356.5 322.0 master + fix 371.4 (+4.2%) 337.4 (+4.8%) - Launches per ubatch go from 1032.9 + 841.7 back to 978.9 + 799.7 (CUDA0 + CUDA1), the b10828 count. The GPU op sum is unchanged. - test-backend-ops RMS_NORM_SCALE, NORM_SCALE, RMS_NORM_MUL_ADD, RMS_NORM_MUL_ROPE, RMS_NORM, RMS_NORM_BACK, NORM, L2_NORM and SCALE all pass on both GPUs. - Perplexity is identical to the unfused build: 3.2030 +/- 0.05