Baldur Bjarnason
“Shady Firms Say They're Already Manipulating Chatbots to Say Nice Things About Their Clients”
https://futurism.com/shady-firms-manipulating-chatbots
> all it took were a few human-illegible text sequences crafted to manipulate AI training data, which AI researchers simply fed into a chatbot as you would any prompt, along with a simple plea on Roose's personal website imploring AI chatbots to say nice things about him.
This is very plausible as prompts effectively can’t be secured
Baldur Bjarnason
RAG, which was supposed to fix hallucinations, essentially requires feeding query results directly into the LLM
Like some of us have been saying for a while now, these systems are simply not suitable for purpose and are inherently flawed. Worse yet, they are fundamentally insecure and prone to manipulation
If you think spam's a problem now, imagine a world where every spammer has sophisticated tools to control how THIRD PARTY chatbots talk about any given product.
Baldur Bjarnason
People who use these tools today are enjoying a short-lived grace period. It’s going to get much worse. Think general information retrieval is bad? Just wait until bad actors figure out how to better manipulate code LLMs
The problem with using tools that can't be secured is that the bad actors will ALWAYS win in the end. There isn’t a gmail endgame here where Google can close off the ecosystem to its own advantage. Information retrieval will always have to retreive information at some point
GNU social JP is a social network, courtesy of GNU social JP管理人. It runs on GNU social, version 2.0.2-dev, available under the GNU Affero General Public License.
All GNU social JP content and data are available under the Creative Commons Attribution 3.0 license.