Nine AI agents fail to meet expert standards on autonomous research benchmark
The 'Into the Unknown' benchmark tasked nine AI agents with solving open research questions from US national labs. None of the 135 submissions met all requirements, with 114 failing integrity checks due to claims of experiments that were never conducted. GPT-6 Astra performed best but still did not meet expert standards.
Summarized by AI from one outlet
Source
- THE DECODERRead original ↗
Latest
- Research
Shanghai Jiao Tong Team Publishes AI-Native Autonomous Molecular Lab in Nature
Researchers at Shanghai Jiao Tong University published a study in Nature introducing AutoDNA, an AI-native scientific lab device. The system enables autonomous molecular biology experiments by co-designing models, experiments, and instruments to overcome challenges in complex workflows and precision.
- Research
Sutton Reaffirms Bitter Lesson: New Methods Needed for LLM Scaling
Turing Award winner Richard Sutton reaffirms that raw compute is insufficient for intelligence, advocating for fundamentally new learning methods. Former OpenAI researcher Danijar Hendrycks agrees, noting that blind transformer scaling has limited scientific thinking, while startups are investing in this gap.
- Research
Comparison of Routing Mechanisms in 10 Open MoE Models
MarkTechPost published a comparison of routing configurations for 10 open MoE models, including DeepSeek, Qwen, and Mixtral. The post details total parameters, active parameters, expert counts, top-k settings, and load-balancing methods.
- Research
WIRED explores world models for robots and autonomous cars
WIRED reports that world models are enabling AI to move from text to 3D spaces by simulating real-world physics. This technology aims to teach robots and autonomous cars how to navigate reality, highlighting its potential in embodied AI applications.
- Research
Alibaba benchmark shows AI coding agents still need human cleanup
Alibaba released a new benchmark called Co-Design Bench to evaluate AI coding agents on real user requests. The results show that even the top performer, GPT-6 Astra, fully solved only 26.4 percent of tasks, contrasting sharply with its over 99 percent score on older benchmarks. The findings highlight significant challenges for AI agents in complex real-world scenarios.
All news → · All rights belong to the original creators