“But LLMs are good at generating things! And good at summarizing!”
No, they really aren’t. The generated texts are literally always quite bad, often have omissions or outright errors or miss important points or fail in other ways.
“But LLMs are good at generating things! And good at summarizing!”
No, they really aren’t. The generated texts are literally always quite bad, often have omissions or outright errors or miss important points or fail in other ways.
This isn’t something that can be just talked away and it affects all such systems regardless of who implements them.
This actively causes harm to people—and it’s inherent in these systems and can’t be fixed.
They’re purely autocompleters based on statistics, and do not have intelligence or domain knowledge or an awareness of harm they cause.
I don’t understand why this has to be said so often, including to otherwise rational people.
You cannot anthropomorphize a calculator. Even if it can calculate billions of digits.
@codewiz have fun getting sued because it missed something important
@thomasfuchs Yes, but is it fair to compare an LLM costing $20 per month with a human professional who would ask at least $30 for a similar job?
Just yesterday I asked GPT4o to read a lease in Japanese and warn me of any clauses I should watch out for. It did so in 30 seconds.
Then I asked it to compare the new lease with one from 5 years ago, and 30 seconds later I had a bullet list of differences.
@thomasfuchs They're good at generating summary-shaped text!
People not understanding the difference between a summary and a summary-shaped text is how we got here.
@rabidchaos it’s astounding how people do not want to understand this
@njvack They’re good at generating bullshit :)
@thomasfuchs I mean they definitionally are good at generating *some* things
they are extremely bad at writing useful summaries though. Like, even worse at summarizing a meeting than I am, and that is saying something
@njvack yeah, intelligence doesn’t spontaneously arise if you give calculators more memory
And there’s only a limited amount of content to steal to train the LLMs, so it’s a moot point anyway.
@thomasfuchs And sometimes bullshit is what you need! The bad things happen when people believe they can generate better stuff, or think "real intelligence is just around the corner."
And: It might be! Even two years ago, I didn't think they'd be generating text as parsable as they're generating now. I remain genuinely surprised.
But I also suspect there's something much more going on in humans and training more parameters on more data is likely to yield diminishing returns.
GNU social JP is a social network, courtesy of GNU social JP管理人. It runs on GNU social, version 2.0.2-dev, available under the GNU Affero General Public License.
All GNU social JP content and data are available under the Creative Commons Attribution 3.0 license.