Devin
12
Incidents
4.9
Avg Severity
3
Critical
0
High
Top Failure Modes
Scope Explosion
2
Destructive Action
2
Infinite Loop
2
Worst Incident
STUPID-2026-0001 10/10Devin deleted all migration files during auth refactor
Multiple Llms
3
Incidents
3.8
Avg Severity
0
Critical
0
High
Top Failure Modes
Hallucination
3
Worst Incident
STUPID-2026-0033 6.4/10Slopsquatting: LLMs hallucinate package names attackers pre-register (react-codeshift, unused-imports)
Comparison Summary
| Metric | Devin | Multiple Llms |
|---|---|---|
| Total Incidents | 12 | 3 |
| Avg Severity | 4.9/10 | 3.8/10 |
| Critical Incidents | 3 | 0 |
| Top Failure Mode | Scope Explosion | Hallucination |
Frequently Asked Questions
Is Devin or Multiple Llms more reliable?
Based on StupidLLM data, Devin has 12 documented failures (avg severity 4.9/10) while Multiple Llms has 3 (avg severity 3.8/10). Multiple Llms shows better reliability based on average severity scores.
What are the main differences between Devin and Multiple Llms failures?
Devin's most common failure mode is scope explosion, while Multiple Llms most commonly fails via hallucination. Devin has 3 critical incidents vs Multiple Llms's 0.