汇集 AI 算力、GPU 加速器、芯片、数据中心与推理基础设施动态。
test-llama-archs : generate dummy test vocab (#29084) Assisted-by: pi:llama.cpp/DeepSeek-V4-Flash-Vision-Exp **Website:** - **Attestations:** - **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b11049/llama-
metal : support qwen4exp hc ops (#29000) Add support for the new DSV4 HC op variants used by qwen4exp: - hc_pre with per-element sigmoid gate (gated variant) - hc_post with identity mixing (comb == nullptr) Assisted-by: pi:llama.cpp/Qwen3.8-27B **Website:** -
CANN跨过拐点，华为补上软件生态
cuda : fix CUB argsort corruption caused by in-place keys (#28389) argsort_f32_i32_cuda_cub called the one-shot DeviceRadixSort::SortPairs API with d_keys_in == d_keys_out (temp_keys, temp_keys). CUB's internal double-buffer ping-pong requires distinct key buf
opencl: add support for bin kernel `flash_attn_f32_f16_bin` (#29046) * opencl: add `flash_attn_f32_f16_bin` * opencl: guarded prefill fa **Website:** - **Attestations:** - **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/rel
# Highlights *713 PRs from 237 contributors.* **New models in this release** (see the [cookbook](https://docs.sglang.io/cookbook) for all supported models): | Model | Type | PRs | Cookbook | |---|---|---|---| | GLM-5.3-Flash | Autoregressive | [#36507](https:/
hexagon: add ROLL op support (#29105) **Website:** - **Attestations:** - **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b11045/llama-b11045-bin-macos-arm64.tar.gz) - macOS Apple Silicon (arm64, KleidiAI e
hexagon: im2col update (#29103) * ggml-hexagon: accept 1D and padded IM2COL ops * ggml-hexagon: make pure-DDR IM2COL kernel is_2D-aware * ggml-hexagon: extend IM2COL DMA patch-embed fast path to 1D * ggml-hexagon: add blocked-staging general IM2COL DMA kernel
Amazon SageMaker AI shipped 13 inference launches in year-to-date across two deployment paths: fully managed endpoints and Amazon SageMaker HyperPod Inference. This post reviews each launch, from inference recommendations and capacity-aware instance pools to t
hexagon: HMX flash-attention head_dim padding (support DK=DV=72) (#26539) Allow HMX flash-attention to run with head_dim not a multiple of 64 (e.g. SigLIP head_dim=72), by operating on DK/DV rounded up to 64 with zero-filled tail lanes. **Website:** - **Attest
You’re deploying a model on a system. It starts up, prompts are getting responses. Now the hard question: Is this fast? Your instincts might lead you to send...
opencl: add bin kernel `kernel_gemm_noshuffle_q6_k_f32_32b_trans_ila_a8_bin` (#28678) * opencl: add A8 Q6_K non-MoE binary kernel * opencl: fix layout compatibility **Website:** - **Attestations:** - **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://githu
Kimi K3 from Moonshot AI is now available on Amazon Bedrock, giving you a powerful new open-weight option for coding and knowledge work. It offers native vision, a 1-million-token context window, and explicit prompt caching to reduce latency and input costs.
协同布局AI基础设施
Today we are announcing the new AgentCore runtime, a capability of Amazon Bedrock AgentCore built for the speed, flexibility, and cost efficiency that production agents demand. It reclaims memory as sessions release it and delivers consistent cold starts regar
ggml : check for allocation failures to prevent crashes (#28149) * ggml : check for allocation failures to prevent crashes * wording **Website:** - **Attestations:** - **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/release
Model-Saver: Write the SWA pattern, 15 more architectures roundtrip (#29042) * llama: read the SWA pattern as a period or a per-layer array Add llama_model_base::load_swa_pattern(), which reads sliding_window_pattern either as one flag per layer or as a period
Amazon SageMaker HyperPod Inference Gateway is a Kubernetes-native, GPU-aware routing add-on for Amazon EKS. It uses real-time GPU signals to send each inference request to the best-suited pod, cutting first-token latency by up to 82% with no changes to your m
With progress that ASML makes with its LPP EUV light sources for its scanners, the company is barely interesting in adopting particle accelerator-based FEL sources. <p>Anton Shilov has been in the PC industry since 1990s playing games, building PCs, and writin
ggml-webgpu: fix supports_op condition for GET_ROWS (#28978) * fix get_rows vec4 handling * Add src strides checking to vec4_aligned of get_rows and the new test case. **Website:** - **Attestations:** - **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://gi
ggml : handle graph buffer reservation failure (#26070) **Website:** - **Attestations:** - **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b11036/llama-b11036-bin-macos-arm64.tar.gz) - macOS Apple Silicon
vulkan: add IQ3_S MMQ matmul kernels (#28822) * vulkan: add IQ3_S MMQ matmul kernels * Make block_a_to_shmem do 2-byte loads (110 bytes is divisible by 2) * Align the check, IQ3_S is also using K tile size **Website:** - **Attestations:** - **macOS/iOS:** - [m
Huawei's mimics Nvidia's approach to AI factories, unveils details about next-generation Ascend NPUs, Kunpeng CPUs, scale-up and scale-out connectivity solutions. <p>Anton Shilov has been in the PC industry since 1990s playing games, building PCs, and writing
### Summary of major features and improvements * #### More GenAI coverage and framework integrations to minimize code changes * New models supported: * On CPU: Gemma-3n * On CPU, GPU: Kokoro-82M, Qwen3-VL-4B with EAGLE-3, Qwen3-ASR, Muse Glimmer 30B, Qwen3.8 2
Every NVIDIA CUDA Toolkit release adds functionality and performance improvements that help developers get more from NVIDIA GPUs and the broader NVIDIA software...
Mixture of experts (MoE) has become one of the defining architectural trends in large-scale AI model training. DeepSeek, Qwen, and Mixtral are examples of MoE...
Amazon Connect Talent is an AI hiring solution built for talent acquisition leaders managing scaled hiring. It delivers AI-led interviews, data-driven assessments, and consistent evaluation, helping recruiters identify strong candidates more efficiently while
Choosing the right vector store for your Amazon Bedrock Knowledge Bases RAG application affects performance and cost. This post compares Amazon OpenSearch Service, Amazon Aurora PostgreSQL with pgvector, and Amazon S3 Vectors across three RAG use cases, with b
Learn how to build a fully serverless pipeline that automatically collects Git metrics from GitHub and GitLab and visualizes them in interactive Amazon Quick Sight dashboards, giving engineering teams near-real-time delivery analytics at low cost.
Wood Mackenzie built APEX, a shared agentic AI platform on Amazon Bedrock AgentCore so every team can ship production agents without rebuilding runtime, identity, observability, and guardrails from scratch. Learn why they chose AgentCore, how APEX Studio opera
