Paul Cantrell
RE: https://unstable.systems/@jneen/116618931097778342
Worth looking at both the quoted text here and •especially• the linked page, which is quite good.
I’ll add another item of my own. The first screenshot mentions giving an LLM the task of “implementing an HTTP server in JavaScript from scratch” in 90 minutes. Sounds impressive, right? Until you remember that every open-source Javascript HTTP server in existence ••was in the training data••.
1/
Paul Cantrell
IT WAS IN THE TRAINING DATA. Your gave the machine a pile of correct answers and free license to plagiarize.
I remember people being wowed that Claude Code could implement a complete C compiler. But somehow it doesn’t sound quite as impressive when you phrase it as “given every existing C compiler as input, the LLM can produce a C compiler as output.”
2/
Paul Cantrell
It’s just •astonishing• how many eye-popping stories of LLMs doing amazing omg-verge-of-magical-superintelligence things turn out to be just unvarnished plagiarism.
In the first months after ChatGPT’s release, I remember a French dept colleague being amazed that GPT could translate and summarize a passage of Le Petit Prince.
It was a lot less impressive when I dug up the 2 or 3 online passages which it had copied almost verbatim and stitched together (sprinkling in a couple of extra words that made it less accurate).
3/
Paul Cantrell
To be fair, competent plagiarism is a nontrivial task. It takes skill for a human to do: finding relevant fragments from an input dataset and stitching those fragments together in a well-formed way is not nothing! It’s impressive and interesting that people have figured out how to make machines do it.
But •that• is not the promise on which investors are valuing the AI megajuggernaut at trillions of dollars.
The reasons plagiarism is cheating in school and malpractice in professional contexts are many of the same reasons that LLMs are not going to replace all knowledge-based human labor.
4/
Paul Cantrell
The reasons we consider plagiarism to be cheating in school and malpractice in professional contexts are many of the same reasons that LLMs are not going to replace all knowledge-based human labor. (Details left as an exercise for the reader.)
And yes, a whole lot of what LLMs do •would• count as plagiarism if one of my students did it manually, and •should• count as plagiarism just the same if they use a machine to do it — not just in a “that’s cheating!!” sense, but perhaps more importantly in a “that’s not really doing the work” sense.
5/
Paul Cantrell
One last example:
The first LLM code example that really made my eyes pop was early after the release of GPT, when somebody got it to combine Breakout with Conway’s Game of Life (a truly delightful idea). It worked!
Funny thing: the Breakout code and the Life code had a •completely• different style and flavor. Red flag. In about 15 minutes of web searching, I was able to find one of the projects (can’t remember if it was the Breakout or the Life half) which it had copied wholesale, with just a few variable renames. And the other half? It was in Python, but it used dictionaries where it really should have used objects — tons of `thing["prop"]` where it should have said `thing.prop`, and lots of other un-Pythonic stuff besides. It was a machine translate of code from another language, very likely Javascript.
The entire thing was a plagiarized Breakout and a plagiarized Game of Life, one transpiled, and all stuck together in a single run loop. To be fair, figuring out how to (1) run both halves of the logic from a single loop and (2) count the Life cells as Breakout bricks is work I'd cheer on from a second-semester intro CS student! It's not, however, quite what's being sold by these companies.
6/
Paul Cantrell
@datarama
I know just enough French to have read Le Petit Prince in the original language (with some struggle), and…it really is beautiful in French in a way that translations don't capture. “On ne voit bien qu'avec le coeur” can translate into English quite directly as “One does not see well but with the heart,” but it just doesn't have the same poetry and magic at all.
Paul Cantrell
As per my posts, I have the luxury of not having LLM vendors shoved down my throat, and I generally avoid them for ethical reasons:
https://hachyderm.io/@inthehands/116581463138461199
But because this all these questions about the usage and limits of these tools keep crashing through my doors, all of our doors, whatever we think of the ethical showstoppers, well…
…fight off amazing percentages of LLM overhype with this one weird question.
/end
Paul Cantrell
@michael_w_busch
Yup. Same with passing the bar.
Michael Busch
Back in 2023, OpenAI was hyping ChatGPT by claiming "it can pass the GRE".
When all it was doing was autocomplete of the answer keys that had been used as input.
Paul Cantrell
@GalbinusCaeli
To be fair, this is an algorithmically difficult problem that was still largely an open question 10-15 years ago! Scale down your expectations by 2 or 3 orders of magnitude, and modern machine learning is truly impressive.
Not a $10 trillion industry. But it's impressive in a “cool research” sense, and also in a “oo, that may pose serious societal danger” sense.
Galbinus Caeli
@inthehands Given pictures of a giraffe, a rhinoceros, an elephant, and a squirrel, find the squirrel. Feel free to reference dictionaries, encyclopedias, nature documentaries and previous responses to this same question.
Paul Cantrell
@joe
Yeah, people were having that same argument about humans and creativity on the more academic side of my circles back in 2023. It would be an interesting one if it didn't have all this investment money weighing it down! (Human learning, both technical and artistic, almost always starts with imitation and repetition; clearly it's a building block of this messy constellation of things that we call “intelligence.”)
I do think the models are getting better at atomizing, as you put it, and I'm disappointed that there's not more research on this family of reverse-mapping problems. One question I've wondered about: can we quantify how much the output depended on a given input? e.g. how would the probability of given output have changed if the model were trained without <pattern> in its training data?
disregard Joe Groff
@inthehands i've noticed a trend in anecdotes recently where people are finding it harder to trace their novel-seeming LLM outputs back to inputs. i wonder if this is a result of them atomizing their inputs more finely, or being "better" at swapping the tokens around to make output look original. (an AI bro might argue that at some point human creativity is doing the same thing…)
Shafik Yaghmour
"HTML parsers in Portland" is another great example
Shafik Yaghmour
The real world measurements of CCC were very bad: https://harshanu.space/en/tech/ccc-vs-gcc/
One of the SQLite benchmarks had a
158,129x slowdown 😱
If you read the blog post in detail:
https://www.anthropic.com/engineering/building-c-compiler
you will see that they could not longer add new features w/o breaking old features. So effectively it was a dead end long before it could become useful in any real world contexts.
Benoît Jones
@janeishly @inthehands @datarama
Love your argument, and an example came to me as I was reading it.
Obelix's dog is called Idefix in the original french, which is funny because idée fixe means stubborn (literally fixed idea). But in English translation the dog is called Dogmatix, which has dogma in it but also of course, the word 'dog'!
Janeishly
@inthehands @datarama No, but if it's a good translation there *should* be other parts in English that are better than the original French. As a translator you can't always capture a particular phrase beautifully, but you can average the beauty over the whole text.
The same applies to humour. Some jokes don't translate but you'll spot an opportunity to get one in elsewhere that wouldn't work in the source language.
Needless to say, machines don't bother doing this.
Paul Cantrell
@huxley @janeishly @datarama
Lovely example! Asterix is a childhood friend, one part of how I learned what little French I know. My dad kept a sticker of Asterix on the monitor of our Apple ][+, to remind him whenever he was getting frustrated at the computer to be like Asterix: optimistic and indefatigable!
Paul Cantrell
@sinabhfuil
The post you are replying to answers the question you're asking.
Petrichor ᚄᚔᚅᚐᚁᚆᚃᚒᚔᚂ
@huxley @janeishly @inthehands @datarama how is idefix stubborn? Idée fixée?
Paul Cantrell
@huxley @janeishly @datarama
Same with Tintin, on both counts.
Even as a kid, I could tell that the stereotype-based humor had an awful stink to it — though that was also an important early education for me in what stereotyping looked like, how colonialism left a cultural footprint on the colonizers, how culture's relationship with alcoholism had changed, etc.
I do find that some (L'Oreille Cassée, L'Affaire Calcule, Tintin au Tibet) still hold up for me, notwithstanding the cultural sour notes.
Benoît Jones
@inthehands @janeishly @datarama Me too, I read them as a child, in both french and English because my mum is french. Also Tintin, though that hasn't aged so well!
Aatch
@inthehands that reminds of when I saw somebody claim to translate the constitution into Japanese on-the-fly (it was obviously a joke/troll). I pasted it into a machine translation tool and it indeed produced the US Constitution... Including the old-style capitalisation of nouns that 1700's English had.
The exact translation was in the training data so it produced the exact translation.
GNU social JP is a social network, courtesy of GNU social JP管理人. It runs on GNU social, version 2.0.2-dev, available under the GNU Affero General Public License.
All GNU social JP content and data are available under the Creative Commons Attribution 3.0 license.