June 22, 2026

AA-Briefcase reveals cost-performance insights for knowledge work models

Last week, a new benchmark called AA-Briefcase was launched to evaluate AI models based on their performance in realistic, long-horizon knowledge work projects. The benchmark, developed by industry experts, necessitates models to handle complex tasks like financial modeling and board presentations over multiple weeks, reflecting actual professional workflows. Notably, open weights models constitute the majority of the cost-performance Pareto frontier identified in the evaluation, highlighting the breadth of available options for effective agentic capabilities. Among the models tested, Claude Fable 5 achieved the highest overall performance, while GLM 5.2 (max) demonstrated a strong cost-efficiency balance, costing significantly less yet scoring closely to more expensive competitors.

Zai: Zai is the organization responsible for developing GLM open weights models. Its GLM-5.2 entry demonstrates competitive agentic performance at lower costs on the new benchmark. Zai contributes to the open weights segment highlighted in the AA-Briefcase analysis.
Kimi: Kimi is a model evaluated on the AA-Briefcase benchmark among various AI systems. It appears in performance distributions across rubric and quality metrics. Kimi provides additional data points on model behavior in long-horizon tasks.
Gemini: Gemini refers to Google’s model line, including Gemini 3.5 Flash, evaluated on the AA-Briefcase benchmark. These models serve as comparators in analyses of agentic performance and cost. Gemini entries help illustrate tradeoffs in the evaluation.
Google: Google develops Gemini models tested on the AA-Briefcase benchmark. Its Gemini 3.5 Flash appears in evaluations of cost tradeoffs against open weights alternatives. Google models contribute to the benchmark’s assessment of diverse AI approaches.
OpenAI: OpenAI develops models including GPT-5.5 evaluated on the AA-Briefcase benchmark. Its entries provide reference points in the leaderboard and cost-performance analysis. OpenAI models participate in the broader comparison of agentic knowledge work capabilities.
GLM 5.2: GLM 5.2 is an open weights model from Zai that ranks as the clear leader among open-weight entries on AA-Briefcase. It delivers strong performance with favorable cost characteristics. The model anchors the open weights presence on the cost-performance frontier.
GLM-5.2: GLM-5.2 is an open weights model from Zai that leads among open-weight entries on the AA-Briefcase benchmark. It offers an attractive capability versus cost tradeoff and scores near top proprietary models. The model exemplifies the competitiveness of open weights approaches in agentic evaluations.
GPT-5.5: GPT-5.5 is an OpenAI model that places in the upper ranks of the AA-Briefcase leaderboard. It participates in comparisons of rubric performance and analytical quality. The model contributes to the benchmark’s assessment of proprietary capabilities.
DeepSeek: DeepSeek is the developer of open weights models featured in AA-Briefcase cost-efficiency comparisons. Its models achieve competitive scores at significantly lower costs than proprietary alternatives. DeepSeek exemplifies the open weights contributions to the benchmark’s Pareto frontier.
MiniMax M3: MiniMax M3 is a model that outperforms relative to its general intelligence ranking on the AA-Briefcase benchmark. It demonstrates strong agentic results in the evaluation. The model highlights varied strengths across different AI systems tested.
Claude Opus: Claude Opus encompasses Anthropic models like Claude Opus 4.8 that rank highly on AA-Briefcase for presentation and overall quality. These models tie or lead in specific Elo categories. They highlight Anthropic’s role in advancing agentic capabilities.
AA-Briefcase: AA-Briefcase is a proprietary agentic knowledge work benchmark that evaluates AI models on long-horizon tasks simulating multi-week professional projects. It requires models to produce realistic deliverables such as financial models, board presentations, and design mock-ups using fragmented context from thousands of source files. The benchmark’s launch highlights the cost-efficiency advantages of open weights models relative to proprietary ones on its cost-performance frontier.
Anthropic AI: Anthropic AI develops proprietary models including those topping the AA-Briefcase leaderboard. Its Claude Fable 5 and Claude Opus 4.8 lead in overall performance and specific quality metrics for knowledge work tasks. The company is central to the benchmark’s demonstration of frontier model capabilities.
Claude Fable: Claude Fable refers to Anthropic’s model family that includes the top-performing Claude Fable 5 variant on AA-Briefcase. It leads in key metrics for agentic knowledge work evaluation. The model family exemplifies proprietary strengths in the benchmark results.
Claude Fable 5: Claude Fable 5 is a leading proprietary model developed by Anthropic. It achieves the top overall score on the AA-Briefcase benchmark through superior rubric pass rate and analytical quality. The model demonstrates strong performance in handling complex, context-heavy knowledge work tasks.
Claude Opus 4.8: Claude Opus 4.8 is a proprietary model from Anthropic noted for its presentation quality on the AA-Briefcase leaderboard. It ranks among the highest-performing models overall while tying for the lead in presentation Elo. Its results underscore Anthropic’s strength in agentic capabilities for knowledge work.
DeepSeek V4 Pro: DeepSeek V4 Pro is an open weights model from DeepSeek that performs strongly on cost efficiency within the AA-Briefcase results. It outperforms certain proprietary models at a fraction of the cost while maintaining competitive Elo scores. Its placement illustrates open weights models’ role on the benchmark’s Pareto frontier.
Gemini 3.5 Flash: Gemini 3.5 Flash is a Google model evaluated on the AA-Briefcase benchmark for cost-performance characteristics. It serves as a reference point for comparing open weights models’ efficiency advantages. The model appears in analyses of tradeoffs between capability and operational cost.
McKinsey & Company: McKinsey & Company provided industry experts who helped design AA-Briefcase scenarios and tasks. Their input ensured the benchmark reflects real-world knowledge work challenges in data science, product management, and corporate strategy. This collaboration enhances the benchmark’s relevance to professional applications.
Boston Consulting Group: Boston Consulting Group contributed experts to the development of AA-Briefcase tasks and scenarios. Their professional experience informed the realistic multi-week project workflows and deliverables tested. The involvement supports the benchmark’s grounding in actual industry practices.

`json
{
“Benchmark Focus”: “AA-Briefcase assesses AI models on complex, long-term projects that are typical in professional environments, emphasizing the need for coherent deliverables over extended periods.”,
“Open Weights Role”: “Open weights models dominate the cost-efficiency frontier in the AA-Briefcase evaluation of agentic capabilities.”,
“Expert Involvement”: “The benchmark’s scenarios were crafted with input from industry professionals who ensured they capture real-world complexities and priorities in knowledge work.”
}
`

AI tools built by Emerald Force

Built and supported by Emerald Force.

You might also like