• Public

  • Public
  • Network
  • Groups
  • Featured
  • Popular
  • People

Screenshot: The researchers started by assuming that there exists a hypothetical bipartite graph that corresponds to an LLM’s behavior on test data. To explain the change in the LLM’s loss on test data, they imagined a way to use the graph to describe how the LLM gains skills. Take, for instance, the skill “understands irony.” This idea is represented with a skill node, so the researchers look to see what text nodes this skill node connects to. If almost all of these connected text nodes are successful — meaning that the LLM’s predictions on the text represented by these nodes are highly accurate — then the LLM is competent in this particular skill. But if more than a certain fraction of the skill node’s connections go to failed text nodes, then the LLM fails at this skill. Source: https://www.quantamagazine.org/new-theory-suggests-chatbots-can-understand-text-20240122/