Notices by Matt McIrvin (mattmcirvin@mathstodon.xyz), page 3

@wizzwizz4 @inthehands It's an interesting problem for this kind of approach--you have certain categories of text in the corpus describing specific events in time, that were appropriate things to say at the time that text was written, but it's now possible to deduce that the text is obsolete. But only if you have an understanding of how time works.

@inthehands @wizzwizz4 I could maybe imagine a mind "embodied" in a virtual world very unlike ours; it would probably develop into a very otherworldly intelligence. But these language models don't even have that--their world is just language interactions severed from direct experience of any of the things the language is referencing.