DeepSeek Open-Sources DeepInfer Inference Framework
DeepSeek announced the open-sourcing of its inference framework DeepInfer, aiming to accelerate on-device inference deployment through deep hardware-software integration. The framework integrates the previously open-sourced TileLang compilation tool, enabling developers to execute AI models more efficiently. This move lowers the barrier for edge AI, allowing lightweight devices to run high-performance inference.
Summarized by AI from one outlet
Source
- 科技新報Read original ↗
Latest
- Open source
Hitch Open Unveiled at CIIE
The open-source physical AI platform Hitch Open was officially released globally at the CIIE, aiming to solve perception and decision-making challenges in physical worlds. An AI robot set a 142-rally world record in table tennis, demonstrating its real-time capabilities.
- Open source
Nous Research Releases Arete-27B
Nous Research released Arete-27B, an Apache 2.0 fine-tune of Qwen3.8-27B. In an 803-prompt eval, propaganda answers fell from 22% to 0.9%, while Hermes Bench scores remained at 65.60%.
- Open source
HN: 31B Audio-Video Diffusion Model Compressed to 14B
A Hacker News user shared an experiment compressing a 31B parameter audio-video diffusion model down to 14B. The model uses INT8 quantization and is reduced to 6 inference steps. This demonstrates potential for reducing model size while maintaining performance.
- Open source
Buoyant Releases Open-Source Inference Control Plane Kinference
Buoyant released Kinference, an Apache 2.0 licensed open-source inference control plane for Kubernetes and vLLM. It features declarative routing, KV-cache-aware load balancing, and reduced median TTFT by 95% in benchmarks.
All news → · All rights belong to the original creators