First Long-Horizon Doc2Repo Training Dataset: Code Agents Move Beyond Bug Fixing and Begin Creating Repositories
With the advancement of LLM Code Agents, the research focus is shifting towards long-horizon, real-world tasks, moving beyond simple bug fixes to full repository generation. To address this, researchers from Renmin University of China introduced the DeNovoSWE dataset. This dataset focuses on long-term software engineering tasks, specifically the "document-to-repository" challenge—generating an entire, executable code repository from a task description.
The DeNovoSWE construction method employs a Divide & Conquer approach. It breaks down target repositories into core capabilities and uses a multi-agent Draft-Critic-Repair workflow to automatically generate high-quality, evaluation-aligned task documents. The dataset also implements difficulty-aware filtering to balance quality and diversity. The result is a high-quality, anti-leakage dataset of 4,818 instances.
Experiments show that models trained on DeNovoSWE achieve significant improvements in long-horizon repository generation. For instance, Qwen3-30B-A3B-Instruct's performance on the BeyondSWE-Doc2Repo benchmark increased from 5.8% to 47.2%, and on NL2RepoBench from 4.3% to 23.0%. Similar gains were observed with stronger backbones, demonstrating that dedicated long-horizon training data is crucial for advancing Code Agents from maintainers to architects capable of planning and building complete software projects from scratch.
marsbit06/25 08:51
