llama.cpp 发布: b11195

来源摘要

ggml-cpu: tiled mul_mat for k-quants (#27851) * Added tiled mul_mat. For each mul_mat_one_chunk, quants are unpacked into (max) 256x256 tiles of int8, one routine per quent. Then microkernel computes 16x16 tiles before writing out 256x256 float reults to main memory. Tests/benches in tests/test-tiled-mulmat.cpp. 3-6x speed improvement for large matmul, break even at 4096x64 * 64x4096, 80% performance (net loss) for GEMV. Error rates trivial (order of 1-e04 max, 1-e05 rmse). * Fixes for ARM/windows builds * more windows fixes, ggml-cpu.h isn't visible in MSVC for some reason * unified iqp + tiled on the Q5_K, IQ4_XS set for benchmarking, updated benchmark * Fixed accidental removal of llama_build_and_test(test-backend-ops.cpp) * First integration of iqp code Co-authored-by Bartowski <3266127+bartowski1182@users.noreply.github.com> * Cleaning up declaration of iq unpacking helpers to align with the bit unpackers * Removed iqp path * Fix cross-platform warnings * Disabling benchmarks unless explicitly enabled * Fix backend_init for DLL-based builds, add self and bartowski to CODEOWNERS for tiled * Put benchmarks behind a flag * kernel fix for AVX2, iq quants * Fix for asan, leaking memory in test-tiled-mulmat and avoid stack use after return * guarding env flags with std::call_once * Simplified repacking for VNNI to a single call per macrotile * No threadlocals anymore, aligned wdata access * Doing aligned reads since we ensure alignment with padding in wdata * Eliminated per-thread gather of Q8_K rows in mul_mat_id, we now gather/repack in a single pass. Repack method now takes pointer array to support both dense/normal and mmid paths. Interface with ggml-cpu.c simplified as a result * Unified/simplified dispatch and support checks. Put details on wdata needed inside the