llama.cpp 发布: b11103

来源摘要

spec : support DFlash for HunyuanOCR (#28890) * model : add DFlash layer-input taps for HunyuanVL DFlash speculative decoding needs the target graph to expose the residual stream entering each layer (res->t_layer_inp[il]) - the draft model reads those tensors to build its cross-context. Qwen3 and the other DFlash-capable targets register them, but the Hunyuan graphs do not, so serving a DFlash draft against a HunyuanOCR target aborts during the first graph build: GGML_ASSERT(t_layer_inp[il] != nullptr && "layer input tensor is null") Register the tensor at the top of the layer loop, mirroring qwen3. The layer input is the residual stream entering layer il, i.e. the output of layer il-1, which is what the draft's target_layers metadata refers to (the converter writes target_layer_ids+1). hunyuan-dense.cpp reuses this graph, so it is covered as well; hunyuan-moe has a separate graph and is untouched. The vector is only read when a speculative implementation enables those layer ids, so there is no behaviour change without a draft model. Tested with tencent/HunyuanOCR 1.5 and its DFlash draft: image requests now run, draft acceptance is ~0.5 and the OCR output is byte-identical to the non-speculative run. Co-authored-by: wendadawen * convert : fix DFlash draft conversion against HunYuan targets Converting a DFlash draft with a HunYuan target failed in two ways. 1. DFlashModel.set_vocab() reuses the target class' vocab handling by calling it unbound with the draft instance, but HunYuanModel.set_vocab() called self._fix_special_tokens(), a method that only exists on HunYuanModel, so the conversion always aborted with AttributeError: 'DFlashModel' object has no attribute '_fix_special_tokens' Make the vocab helpers module-level functions taking the model explicitly, so they do