Transformers 发布: Release 5.17.0

来源摘要

# Release v5.17.0 ## New Model additions ### HYV4 Hy4-Preview is a 780B-parameter mixture-of-experts language model that activates 49B parameters per token. Each MoE layer holds 256 routed experts plus one always-active shared expert and routes every token to 8 of them. The context window is 1M tokens. The architecture combines four features: - **Multi-head Latent Attention (MLA)** compresses keys and values into a low-rank latent (`kv_lora_rank`) that `kv_b_proj` expands back to one key/value per query head. - **DeepSeek Sparse Attention (DSA)** selects `index_topk` keys per query with a lightweight indexer. Following [IndexShare](https://huggingface.co/papers/2603.12201), only the layers marked `"full"` in `indexer_types` run an indexer; `"shared"` layers reuse the previous full layer's selection. - **Gated MLA with learnable attention sinks**, where each head owns a sink logit that participates in the softmax and contributes no value, as in [GPT-OSS](./gpt_oss). - **Independent Hyper-Connections (iHC)** replace the plain residual path with `hc_mult` parallel residual streams that are collapsed before, and redistributed after, every sublayer. The implementation does not execute the multi-token prediction (MTP) layers. Released checkpoints keep those weights so that other runtimes can use them for speculative decoding; they are ignored at load time. **Links:** [Documentation](https://huggingface.co/docs/transformers/main/en/model_doc/hy_v4) * Add h4 (#48473) by @ArthurZucker in [#48473](https://github.com/huggingface/transformers/pull/48473) ### VibeVoice [VibeVoice](https://huggingface.co/papers/2508.19205) is a novel framework for synthesizing high-fidelity, long-form speech with multiple speakers by employing a next-token diffusion approach within a Large Language M