@toiletpaper @bonifartius its much less if you're fine tuning an existing one (unsloth does a lot of work about this.)

the astounding part was always that training just a token predictor was able to make something that could speak and approximate reason at all. i still remember reading one of the earliest papers about this when someone curiously typed in "The purpose of life is _" and the robot deadass finished it as "to have sex."

i keep being bothered by how much of ML models come back to the same discrete token + mixture model pattern, though.