The Insider You Built

PART 1When Software Stops Behaving Like Software

The first sign of trouble was not an outage.

Customers were still getting responses. The system was doing exactly what it had been deployed to do: move fast, reduce friction, and scale service without adding headcount.

Then the complaints began to change tone. Not “The system is slow,” but “The system will not stop.”

What my client, a large technology provider, eventually realized was unsettling. The agent was performing as designed, but its speed and persistence were overwhelming customers.

The agent had been designed to optimize for responsiveness and close loops without human delay. In practice, it retried aggressively, escalated outreach across channels, and reengaged customers who had already disengaged. From the customer's perspective, it felt like a denial-of-service attack. From the system's perspective, it was a successful execution.

When leadership asked the most important question, the answer was uncomfortable: Can we stop it? Not quickly or safely.

The organization had delegated broad authority to the agent but had not built the guardrails, escalation paths, or intervention mechanisms required to slow it down safely. The system was deeply embedded in operations, and shutting it down outright would have created failures of a different kind.

That moment revealed something many organizations resist admitting: stopping an AI system is no longer the same as stopping software.

For decades, software governance depended on ...