In a significant advancement for artificial intelligence in software engineering, researchers have developed MirrorCode, a benchmark that allows AI models to autonomously code over extended periods, tackling complex tasks that would typically require human engineers several weeks to complete. Notably, AI model Claude Opus 4.7 successfully reimplemented a 16,000-line bioinformatics toolkit, finishing the task in just 14 hours and at a cost of $251, showcasing the model’s efficiency compared to the estimated 2–17 weeks it would take a human. MirrorCode, co-developed with METR, aims to evaluate AI’s capabilities in long-horizon coding tasks, focusing on specifying detailed behavioral outcomes and encouraging open research by releasing 22 of its 25 target programs as open source.

METR: METR is an organization specializing in the evaluation of advanced AI systems, with a focus on capabilities assessment and safety implications. It co-developed the MirrorCode benchmark and provided grant support for the project. This partnership positions METR as a key collaborator in creating standardized tests for long-horizon AI software engineering performance.
MirrorCode: MirrorCode is a long-horizon software engineering benchmark that evaluates AI models on autonomously reimplementing entire programs from specifications, documentation, and tests without access to original source code. The benchmark emphasizes cheat resistance through sandboxing, held-out tests, and execute-only environments across domains like bioinformatics, interpreters, and cryptography. In this news, MirrorCode is introduced as the central tool developed to measure AI progress on extended coding tasks that can span days or weeks of autonomous effort.
Claude Opus 4.7: Claude Opus 4.7 is an advanced AI language model from the Claude series developed by Anthropic. It achieved leading results on the MirrorCode benchmark by successfully handling complex reimplementation tasks across multiple programs. The news highlights its performance as evidence of current AI capabilities on extended autonomous coding challenges.

`json
{
“AI Evaluation”: “MirrorCode allows for comprehensive evaluations by enabling larger inference budgets, reflecting the extensive nature of software engineering tasks comparable to human-level work.”,
“Open Research”: “The initiative provides open access to its scaffold and a majority of its target programs, while retaining some as a private test set to facilitate ongoing research in long-horizon AI coding capabilities.”,
“Benchmark Design”: “MirrorCode emphasizes tasks that are challenging but achievable for AI to fully execute, with a focus on adhering to specific requirements over addressing undocumented features.”
}
`