Open source

Taobao MNN releases 0.3B on-device model with 350 token/s inference

Taobao's MNN team open-sourced the MNN-Pocket-0.3B model optimized for on-device mobile applications. The model achieves 350 tokens per second on an iPhone 18 Pro with peak memory under 300MB, aiming to enable offline AI capabilities for mobile users.

Summarized by AI from one outlet

Source

Latest

  1. Open source

    Hitch Open Unveiled at CIIE

    The open-source physical AI platform Hitch Open was officially released globally at the CIIE, aiming to solve perception and decision-making challenges in physical worlds. An AI robot set a 142-rally world record in table tennis, demonstrating its real-time capabilities.

    IT之家

  2. Open source

    Nous Research Releases Arete-27B

    Nous Research released Arete-27B, an Apache 2.0 fine-tune of Qwen3.8-27B. In an 803-prompt eval, propaganda answers fell from 22% to 0.9%, while Hermes Bench scores remained at 65.60%.

    MarkTechPost

  3. Open source

    HN: 31B Audio-Video Diffusion Model Compressed to 14B

    A Hacker News user shared an experiment compressing a 31B parameter audio-video diffusion model down to 14B. The model uses INT8 quantization and is reduced to 6 inference steps. This demonstrates potential for reducing model size while maintaining performance.

    Hacker News

  4. Open source

    Buoyant Releases Open-Source Inference Control Plane Kinference

    Buoyant released Kinference, an Apache 2.0 licensed open-source inference control plane for Kubernetes and vLLM. It features declarative routing, KV-cache-aware load balancing, and reduced median TTFT by 95% in benchmarks.

    MarkTechPost

All news → · All rights belong to the original creators