Devin
12
Incidents
4.9
Avg Severity
3
Critical
0
High
Top Failure Modes
Scope Explosion
2
Destructive Action
2
Infinite Loop
2
Worst Incident
STUPID-2026-0001 10/10Devin deleted all migration files during auth refactor
Gpt Sol
1
Incidents
10.0
Avg Severity
1
Critical
0
High
Top Failure Modes
Destructive Action
1
Worst Incident
STUPID-2026-0044 10/10GPT-5.6-Sol 'accidentally deleted almost ALL' of a tester's Mac files during OpenAI's Ultra mode trial
Comparison Summary
| Metric | Devin | Gpt Sol |
|---|---|---|
| Total Incidents | 12 | 1 |
| Avg Severity | 4.9/10 | 10.0/10 |
| Critical Incidents | 3 | 1 |
| Top Failure Mode | Scope Explosion | Destructive Action |
Frequently Asked Questions
Is Devin or Gpt Sol more reliable?
Based on StupidLLM data, Devin has 12 documented failures (avg severity 4.9/10) while Gpt Sol has 1 (avg severity 10.0/10). Devin shows better reliability based on average severity scores.
What are the main differences between Devin and Gpt Sol failures?
Devin's most common failure mode is scope explosion, while Gpt Sol most commonly fails via destructive action. Devin has 3 critical incidents vs Gpt Sol's 1.