Screenshot: Nonetheless, Hinton thinks the work lays to rest the question of whether LLMs are stochastic parrots. “It is the most rigorous method I have seen for showing that GPT-4 is much more than a mere stochastic parrot,” he said. “They demonstrate convincingly that GPT-4 can generate text that combines skills and topics in ways that almost certainly did not occur in the training data.” (We reached out to Bender for her perspective on the new work, but she declined to comment, citing a lack of time.) Source: https://www.quantamagazine.org/new-theory-suggests-chatbots-can-understand-text-20240122/
https://cdn.masto.host/daircommunitysocial/media_attachments/files/111/802/009/441/472/197/original/da8dfa442b1159a3.png