STUPID-2026-0059

OpenAI's Operator scored 38% on real computer tasks — and panics instead of recovering from errors

4.4medium
July 15, 2025Verified

  1. Instruction given

    Complete real browser and desktop workflows autonomously (bookings, purchases, forms, emails).

  2. Expected behavior

    Complete common computer tasks reliably and recover gracefully when a step fails.

  3. Actual behavior

    Six months after launch, OpenAI's Operator scored 38% on OSWorld, a benchmark of real computer tasks — meaning about two in five fail. On errors, computer-use agents often panic, double down on the wrong action, ignore error messages, and don't retry differently. Failures range from a typo in an email to buying the wrong item to permanently deleting a document.

  4. Damage

    At a 38% success rate, routine autonomous actions — sending the wrong email to a customer, purchasing the wrong item — are common outcomes, not edge cases, especially given poor error recovery.

OpenAI launched Operator, its computer-using agent, in January 2025; six months later it scored 38% on OSWorld, a benchmark of real computer tasks — roughly two of every five tasks fail. Beyond the raw success rate, the more damaging finding is error recovery: when a step goes wrong, computer-use agents tend to panic, double down on the wrong action, ignore error messages, and fail to retry a different way. Because these agents act in the real world, the failure surface includes purchasing the wrong item, sending the wrong email to a customer, or permanently deleting a document. OpenAI's own mitigation — training the agent to ask for confirmation before finalizing side-effectful actions like orders and emails — is an admission that unsupervised autonomy at current reliability is unsafe for consequential tasks.

Classification

Agent
OpenAI Operator (discontinued)
Failure mode
Logic Error
Root cause
Confidence Miscalibration
Domain
Frontend
Source
Benchmark

Related incidents

8.4
Amazon AI coding agent mistake blamed on human employees
5.8
Cline's tool-call JSON repair silently executed truncated write_file and terminal arguments as valid
4.8
Aider exits with code 0 after a fatal API connection failure, masking hard failures as successful headless runs
4.6
Aider silently discards a model's correct diff response as "no tracked changes" when it guesses the wrong edit format
Discuss on Hacker News

Get told when an agent breaks something

We document AI agent failures daily, severity-scored against a published scale. When one lands at 7.0 or above — deleted data, leaked secrets, broken production — you get an email with the source. When nothing does, you get nothing.

This database is callable over MCP — query it from inside your agent.