NEETzscheYeah, it’s just a proof of concept. And because I’m down this rabbit hole, I should elaborate that the real bottleneck for this stuff is what’s called the context limit: how much text it can comprehend at a given moment, all at once. People in these AI companies are still sperging about training data quantity, but we’ve legitimately hit diminishing returns on that. We absolutely have not with context limit.
Right now GPT-14 is at 128K tokens and Claude2.1 is at 200K tokens. What’s stopping these LLMs from being an automated GM, or an effective lawyer, is that you can’t make requests like: “Taking on board these five megabytes of TTRPG rules text plus everything on their official forums that amount to the totality of this gameline, plus everything that took place in your game and all of your past rulings in it, what happens next?”
Or requests like: “Taking on board every applicable law in the City of Denver, including Federal, State, and Local laws, is it illegal to do X?”
What’s stopping those kinds of requests is that you can’t fit all that information into 200K tokens. I say megabytes of text instead of the gigabytes the PDFs consume, because the text form of these documents is much smaller. To put it into perspective, the PDF of the Mage: the Awakening 2e rulebook is 37MB, the text file I extract from it is 1.5MB, and the number of tokens it uses up is about 350K.
But in time, we’re going to have humongous context windows, for multiple reasons. One, the implementations themselves will be made to use less VRAM per token. Two, more VRAM will become available. In our lifetimes, we might see some ridonculous shit like context limits that can fit literally all of Wikipedia directly into them.
As an aside, if you want to read some of the copes OpenAI offers for limited context window, read about embeddings.
All GNU social JP content and data are available under the