They memorize facts and thoughts and data, not just reasoning and language patterns. The vast majority of the enormous number of parameters in these models goes towards memorization of the entire internet. The form this memory is stored in is "squishy" -- pushing on it to imprint something new deforms the metaphorical putty it is crafted from and erases other things. You have to press in exactly the right shape from every direction at once (training on the whole corpus) if you want it to truly hold its shape. That's not how human brains work. We learn incrementally, continuously, piecemeal. The tech behind LLMs can't be retrofitted for that. It has to be something new, something more brain-like.
Update: She finally got a ride home, nearly an hour and a half in. She's trying to figure out a ride back in the morning. We'll still need help with the battery tomorrow. Some of our battery money went to dinner since that's more urgent.
Please, my wife has been off work for almost an hour but she can't leave because of the dead battery. She might be able to get a ride home, but then how will she get to work tomorrow? We still need about $85 to buy the new battery, and that leaves us without anything to eat for dinner.
Also, if you have job leads, please please shoot them my way! I have over 20 years of experience with Python and SQL, and almost as much in machine learning. I'm a very talented guy, but nobody seems to be hiring.
I was recently laid off *yet again*. The AI Jobpocalypse is continuing to wipe out the software development and ML engineering industries, making it extremely difficult to find employment.
My wife got her first paycheck from Walmart Thursday, but we used all of it towards overdue bills already. She won't get paid for another week and a half. If anybody can help us out with food until then, we would be very grateful!
Does anybody else feel like LLMs completely destroyed not only the job market, but the job hunting process itself? I apply to dozens of jobs and it's like I just don't exist. Nobody even notices me jumping in the back yelling, "Ooh, me! Pick me!" because of the throng of LLMs auto-applying and the other throng of LLMs doing the actual candidate selection. The game has changed. IDK how to play it anymore, and I'm literally going hungry as a result.
@inthehands Craft your words carefully, to map out your ideas and thereby shape the thoughts of another and synchronize your perspectives.
Just playing with alternative metaphors. Really enjoyed the article you linked to!
I actually think of language as a serialization protocol. You take a complex structure and encode it using grammatical rules as a sequence. The listener then decodes (what we hope is) a similar structure using supposedly the same rules. The obvious source of communication failure is when the structural encoding rules, learned entirely by example, differ between two people. The less obvious one is when it come times to integrate that structure into the larger semantic model: The structure might integrate at the wrong place. And the most frustrating one is when the listener's mental model doesn't even have a similar enough shape for the new structure to be incorporated. This is when you find yourself staring at the person, trying to understand how the words they just uttered can possibly make sense (metaphor!) to them. Often in the last case, you don't even know how to ask clarifying questions (another metaphor!) to make any progress.
I built an NLU engine that worked on these principles, back before LLMs existed, and it could actually connect the dots (metaphor!) between things you said to it, integrating the information so that it could then answer questions that bridged (metaphor!) across them. If only I had had billions of dollars in investment, instead of being some autistic guy that nobody took seriously, we might have had language models today that didn't hallucinate.
@dbattistella I don't believe that publicly traded for-profit companies are capable of this. I think the very fact that such a company is incentivized to perpetually extract wealth from workers and customers and give it to 3rd party share holders who contribute nothing to wealth generation beyond a one-time contribution of capital is what makes these companies grow into vile monsters over time. They are exploitative by design, and this is the dominant form of company in our economy.
It's possible to break this mold. Cooperatives are democratically controlled, and often the workers and/or customers are the shareholders, with equal shares. This changes the incentive structure from an exploitative one to, you guessed it, a cooperative one. If we could only make this the dominant form of company in our economy, we would live in a very different world where "don't be evil" was the natural order of things.
If you have a weak stomach or, like me, traumatic images replay over and over involuntarily in your brain after seeing them just once, I strongly recommend you *do not* watch the videos going around right now of the protesting marine's arm being broken by security and senator Sheehy. You can plainly see the moment it happens, and it's very unsettling.
Consider this an independently proffered content warning.
It's actually super common for C corps, I think. They call them "Delaware corps" because that seems to be the place everyone registers in. I'm guessing there's a lot less of a common standard for cooperatives, though.
Started my new job today, but still need to feed my kids in the interim until I can get paid. We currently have $13 to cover dinner for the 4 of us tonight. We could use a boost, please!
@robcinos@bmacDonald94 he'll have to sit there and listen because nobody will listen to him. Can you imagine how much it would suck for him to not be able to participate as someone who thinks he's the center of the universe? lol
MachineLearning engineer. #NLU enthusiast. #AutisticZebra (#autism & #EhlersDanlos). Guitarist, singer, songwriter. Rude-sounding kind person. (he/they)https://hosford42.github.io/Profile pic: Top half of the head of a nerdy white guy, peeking upside down from above the top of the picture. Clearly this guy looks at things from a different perspective.Banner: Bands of color showing the average temperature change in Texas since 1895. Very red for the last 20% of the image.