In other words, a stopped clock is always wrong.
It doesn't matter that it will coincidentally display the correct time twice per day. That coincidence doesn't have any connection to a clock's function as a tool. It's not doing the thing you think clocks do. It's not right about the time any more than a photo of a clock is. Or a description of a clock in a book. A stopped clock isn't a time keeping device. It's something else, that's merely shaped like a time keeping device.
LLMs aren't thinking. They aren't learning, understanding, or remembering. They're doing something else, that is merely shaped like thinking. They are a stopped clock, and they are always wrong.