CUGA Apps
🚀
59
2 dozen real-life agents optimized for open models & stack
Enterprise Agents and Benchmarks
Enterprise agent ecosystem featuring AssetOpsBench (industrial) and ITBench (SRE, FinOps, CISO), CUGA to accelerate AI Automation
-
-
AssetOpsBench
🚀21Benchmark asset operation performance in the browser
-
CUGA Agent
🤖105Configurable Generalist Agent, leader in AppWorld Benchmark
-
MCP-Cosmos: World Model-Augmented Agents for Complex Task Execution in MCP Environments
Paper • 2605.09131 • Published • 60 -
VAKRA Leaderboard
🏆19Explore AI agent benchmark and submit results
-
ibm-research/AssetOpsBench
Viewer • Updated • 467 • 681 • 43 -
ITBench-Lite-Space
🚀8Develop and run interactive code notebooks with JupyterLab
-
Survey on Evaluation of LLM-based Agents
Paper • 2503.16416 • Published • 97 -
ibm-research/ITBench-Lite
Updated • 4.48k • 12 -
ITBench: Evaluating AI Agents across Diverse Real-World IT Automation Tasks
Paper • 2502.05352 • Published • 2 -
AssetOpsBench: Benchmarking AI Agents for Task Automation in Industrial Asset Operations and Maintenance
Paper • 2506.03828 • Published • 20 -
ibm-research/ITBench-Trajectories
Updated • 282 • 4 -
From Static Templates to Dynamic Runtime Graphs: A Survey of Workflow Optimization for LLM Agents
Paper • 2603.22386 • Published • 57 -
ScarfBench
🐠1Java framework migration
-
SPIRAL: Symbolic LLM Planning via Grounded and Reflective Search
Paper • 2512.23167 • Published • 1 -
Results and Retrospective Analysis of the CODS 2025 AssetOpsBench Challenge
Paper • 2605.08518 • Published • 11 -
ibm-research/VAKRA
Viewer • Updated • 1.33k • 580 • 46 -
General Agent Evaluation
Paper • 2602.22953 • Published • 12 -
ibm-research/ScarfBench
Updated • 557 • 19 -
lyuchao/ITBench-Trajectories
Updated • 24 • 1 -
Beyond Static Leaderboards: Predictive Validity for the Evaluation of LLM Agents
Paper • 2606.19704 • Published • 41