Restore LLM Inference Capacity in Seconds with Shadow Engine Recovery in NVIDIA Dynamo
When an LLM engine process fails, the standard recovery path involves a cold restart. This requires loading weights into HBM from storage, compiling kernels,...
接入 API · 个人 AI 解读连接自己的模型解读资讯，浏览新闻无需配置。
本地 AI 部署
关注本地模型部署、推理引擎、量化和工作站运行工具。
120 条资讯按来源发布或更新时间排序
Open WebUI 发布: v0.11.1
### Added - 🚦 **Human in the loop tool approval.** Where an administrator has turned it on, you can switch a conversation from letting tools run freely to being asked first, so a model that wants to use a tool stops and waits for you to allow or deny it, one
Jalapeño’s first results show industry-leading speed and efficiency in AI inference
Jalapeño is a custom inference chip from OpenAI that delivers faster, more power-efficient AI inference, with higher throughput and lower latency for modern models.
SGLang 发布: v0.5.18
# Highlights *710 PRs from 212 contributors.* **New models in this release** (see the [cookbook](https://docs.sglang.io/cookbook) for all supported models): | Model | Type | PRs | Cookbook | |---|---|---|---| | Muse Glimmer | Autoregressive (Multimodal) | [#34
ONNX Runtime 发布: ONNX Runtime WebGPU Plugin EP v0.3.0
ONNX Runtime WebGPU Plugin EP 0.3.0 expands model and data-type coverage, improves generative-model performance, and strengthens configuration, reliability, and release tooling. These release notes were drafted with AI assistance. ## Highlights ### Model and o
NVIDIA Vera Rubin and Blackwell Set a New Standard for Agentic AI Performance per Watt
AI agents have expanded inference from single-turn interactions into multi-step workflows that reason, invoke tools, coordinate subagents, and carry growing...
Ollama 发布: v0.32.15
## What's Changed * New desktop onboarding flow on first launch * Caches resolved model metadata between requests, cutting time-to-first-token by roughly half (TTFT dropped from ~995 ms to ~524 ms in benchmarks) * Fixes a bug where chat and generate could wedg
Co-Designing AI Model Attention for Fast, Interactive Long-Context Inference
As agentic and long-context workloads become common, the context lengths increase and attention consumes a larger share of inference time (Figure 1). Because...
ONNX Runtime 发布: ONNX Runtime v1.28.1
This is a patch release on top of [v1.28.0](https://github.com/microsoft/onnxruntime/releases/tag/v1.28.0), containing support for device-free WebGPU compilation, improved compatibility with sandboxed Windows processes, and targeted graph-validation fixes. ##
ONNX Runtime 发布: ONNX Runtime CUDA Plugin EP 0.1.0
This is the first release of ONNX Runtime CUDA Plugin EP, providing CUDA execution as a separately packaged plugin execution provider. These notes cover commits affecting CUDA Plugin EP core code, CMake integration, and its primary build and package pipeline.
Ollama 发布: v0.32.14
## What's Changed * llm: transcode WebP images for llama-server * renderers/qwen: tolerate non-leading system messages **Full Changelog**: https://github.com/ollama/ollama/compare/v0.32.13...v0.32.14
ONNX Runtime 发布: ONNX Runtime v1.29.0
## Announcements & Breaking Changes - onnxruntime-web has announced the deprecation of WebGL and JSEP. The native WebGPU EP is the recommended path going forward. See the deprecation and migration plans for details ([#29716](https://github.com/microsoft/onnxru
vLLM 发布: v0.27.1
This is a patch release on top of v0.27.0. - Support quantized DSpark Markov heads (#50424)
Intel OpenVINO 发布: 2026.3.0
### Summary of major features and improvements * #### More GenAI coverage and framework integrations to minimize code changes * New models supported: * On CPU, GPU, and NPU: SmolLM3-3B, LFM2-1.2B, LFM2.5-1.2B * On CPU and GPU: Harrier OSS-v1-0.6B, Qwen3-8B wit
SGLang 发布: v0.5.17
# Highlights *582 PRs from 194 contributors.* **Kimi K3 day-0 support**: A 2.8T-parameter multimodal LatentMoE (896 experts, top-16, routed in a 3584-dim latent space) with a 1M-token context, 69 KDA linear-attention layers interleaved with 24 MLA layers, and
vLLM 发布: v0.27.0
# vLLM v0.27.0 Release Notes ## Highlights This release features 561 commits from 242 contributors (64 new)! * **Kimi K3 support** with a full stack landing in one release: core model files and kernels (#50089, #50000), Python (#50093) and Rust (#50104) fronte
AMD Acquires Taalas to Advance Compute Solutions for Rapidly Growing AI Inference Market
来源已更新，查看详情及原始发布记录。
ONNX Runtime 发布: ONNX Runtime WebGPU Plugin EP v0.2.1
## Highlights - **Major performance work for attention-heavy LLMs.** - FlashAttention decode kernels were fused and extended for any sequence length ([#28389](https://github.com/microsoft/onnxruntime/pull/28389)). - FlashAttention prefill shared-memory path wa
Open WebUI 发布: v0.11.0
### Added - 🎨 **Redesigned interface.** Open WebUI has been visually rebuilt from the ground up. All aspects of the User Interface, from the chat view to the admin panel. Now with a narrower conversation column, lighter typography, tidier spacing, consistent
vLLM 发布: v0.26.0
# vLLM v0.26.0 Release Notes ## Highlights This release features 411 commits from 212 contributors (61 new)! * **New Inkling model family** with a full support stack: base modeling (#48799), piecewise CUDA graph support (#48822), Hopper FA4 relative attention
ONNX Runtime 发布: ONNX Runtime v1.28.0
## Announcements & Breaking Changes - Upgraded to **ONNX 1.22.0** and protobuf 6.33.5 ([#28754](https://github.com/microsoft/onnxruntime/pull/28754), [#29606](https://github.com/microsoft/onnxruntime/pull/29606), [#28967](https://github.com/microsoft/onnxrunti
SGLang 发布: v0.5.16
# Highlights *574 PRs from 169 contributors.* **DSpark: confidence-driven speculative decoding**: A new speculative algorithm. It drafts semi-autoregressively in blocks, then sizes each verify window from the draft's own confidence instead of a fixed draft len
AMD and Cerebras Announce Industry-Leading Ultra-Low-Latency and High Throughput AI Inference Solution
来源已更新，查看详情及原始发布记录。
AgentScope 发布: v2.0.5
## Highlight **Agent** - Support structured output (#2150) - Support environment information injection (#2134) **Workspace** - Support OpenSandbox (#1953), Daytona (#1943), K8s (#1933), Bubblewrap (#2051) **Tool** - Support PowerShell for Windows (#2132) **RAG
CVE-2026-5241: huggingface/transformers: Arbitrary Code Execution During Model Initialization in the LightGlue Model Loading Path
A vulnerability in the LightGlue model loading path of huggingface/transformers version 5.2.0 allows an attacker-controlled model repository to execute arbitrary code during model initialization. The issue arises because the `trust_remote_code` parameter, inte
AgentScope 发布: v2.0.4.post1
## What's Changed * fix(gemini): fix gemini schema containing null by @qbc2016 in https://github.com/agentscope-ai/agentscope/pull/2020 * feat(workspace): add K8sWorkspace with Pod/PVC lifecycle, tar-stream backend and MCP gateway support by @XiuShenAl in http
AMD ROCm 发布: ROCm 7.14.0 Release
# ROCm Core SDK 7.14.0 release notes ROCm Core SDK 7.14.0 transitions ROCm to [TheRock](https://github.com/ROCm/TheRock), a build and release system that introduces a modular architecture to improve flexibility, maintainability, and alignment with community us
Transformers 发布: Release v5.14.0
# Release v5.14.0 ## New Model additions ### Inkling (fresh from Thinking Machines): 975B total, 41B active * Add Inkling model #47347 by @molbap @Cyrilvallez @eustlb and @zucchini-nlp Inkling is a general-purpose multimodal model that accepts text, image and
vLLM 发布: v0.25.1
# vLLM v0.25.1 ## Highlights This release features 2 commits from 2 contributors (1 new)! v0.25.1 is a patch release containing two targeted bug fixes on top of v0.25.0. ### Bug Fixes * **Avoid blocking model launching when no system FFmpeg is available for To
SGLang 发布: v0.5.15.post1
v0.5.15.post1 includes a few patches, mostly for GLM 5.2 - #30454 #30627: Fix DSA model launching on non Cuda/HIP devices - #30858: Fix flashinfer dependency on Cuda 12 images - #31001: Fix NaN outputs caused by flashinfer trtllm FP4 MoE kernels on long input