Embed Notice

Let me preface my comment about the article below with a full disclosure: I work for a Microsoft subsidiary, and I use GitHub Copilot on a regular basis. And no: I'm not required or forced to do so, neither measured by how much I use it. The words in this post (and this profile) are my own.

As much as I don't like the argument that this failure was a user error, not an AI error, I do agree that the user letting the AI do the work by itself was an error.

I'm not naive. I know that an AI/LLM will make mistakes, and for that reason I don't let them have "write" access to anything that can be harmful (to me or anyone else). I use Devcontainers, which are isolated from my OS. I manually approve any local commands, and thoroughly review what they are supposed to do. If something needs API access (like an MCP agent), they get a read-only key, and even so, they need to ask me before taking any "read" actions. I give it strong guardrails, boundaries, limits and strive to be as clear with my instructions as possible (even if it requires lots of verbosity).

If you let the AI modify your stuff, and it does it wrong, yes, it was the AI's fault, because it made the wrong choices. But you, as the user, are equally responsible for it when you allow the AI to do things using your name/login/role/environment by itself (or "autonomously", as they like to say).

"Trust but verify". Words to live by.

https://www.pcmag.com/news/amazon-links-2-aws-outages-to-autonomous-kiro-ai-coding-agent