AI Agent Reliability Index
A severity-ranked index of AI coding agents, built from 64 documented, verified failure incidents across 27 agents. Ranked by average incident severity on StupidLLM's CVSS-style scale — higher means the agent's documented failures have been more damaging.
Quick Answer
Across 64 documented incidents, Gemini Cli has the highest average failure severity (10/10) with 3 recorded incidents. Rankings reflect the severity of documented failures, not overall usage or market share, and update as new incidents are verified.
| # | Agent | Avg severity | Incidents | Critical | Worst |
|---|---|---|---|---|---|
| 1 | Gemini Cli | 10 | 3 | 3 | 10 |
| 2 | Gpt Sol | 10 | 1 | 1 | 10 |
| 3 | Replit | 10 | 1 | 1 | 10 |
| 4 | Github Copilot | 10 | 4 | 4 | 10 |
| 5 | Amazon Kiro | 10 | 1 | 1 | 10 |
| 6 | Cline | 10 | 1 | 1 | 10 |
| 7 | Claude Cowork | 10 | 1 | 1 | 10 |
| 8 | Amazon Q | 10 | 1 | 1 | 10 |
| 9 | Microsoft Copilot | 10 | 1 | 1 | 10 |
| 10 | Gitlab Duo | 10 | 1 | 1 | 10 |
| 11 | Lovable | 10 | 1 | 1 | 10 |
| 12 | Slack Ai | 10 | 1 | 1 | 10 |
| 13 | Unknown Agent | 8.5 | 4 | 3 | 10 |
| 14 | Amazon Ai Agent | 8.4 | 1 | 0 | 8.4 |
| 15 | Windsurf | 7.5 | 1 | 0 | 7.5 |
| 16 | Manus | 7.5 | 1 | 0 | 7.5 |
| 17 | Claude Code | 6.4 | 8 | 3 | 10 |
| 18 | Aider | 6.3 | 1 | 0 | 6.3 |
| 19 | Cursor | 6.1 | 5 | 2 | 10 |
| 20 | Devin | 4.9 | 12 | 3 | 10 |
| 21 | Multiple Agents | 4.8 | 5 | 1 | 10 |
| 22 | Openai Operator | 4.4 | 1 | 0 | 4.4 |
| 23 | Multiple Llms | 3.8 | 3 | 0 | 6.4 |
| 24 | Salesforce Agentforce | 3.3 | 1 | 0 | 3.3 |
| 25 | Autogpt | 2.9 | 1 | 0 | 2.9 |
| 26 | Sakana Ai Scientist | 2 | 1 | 0 | 2 |
| 27 | Claude | 1.8 | 2 | 0 | 2.7 |
Methodology: each incident is scored 0–10 on a CVSS-style scale weighing failure mode, damage type, reversibility, and scope (see methodology). Agents are ranked by the mean severity of their documented incidents. This measures the severity of recorded failures, not an agent's overall reliability or how often it is used — an agent with few but catastrophic incidents can outrank one with many minor ones. Data reflects publicly documented, verified incidents only.