Alex Gaynor considers the problem that even if they are maybe sometimes good at some things, LLMs are not always good at everything, and the fact that we have no way to tell the difference severely limits their utility: https://alexgaynor.net/2025/mar/05/generality/