Embed Notice

In theory the embedding can be passed in as context to most LLMs, then “summarize x in ten words” can be weapped around it, but I’m curious if there’s any way to just poke deeper at the underlyinf model’s embedding/token relationships.

From what I’ve been reading, that was easier in earlier models like GPT-2, but later ones aren’t symmetrical. Ie, word -> tokens -> embedding -> tokens -> words wouldn’t arrive back at the same place.