One of the most laughable things about the LLM bubble has been the way it ignores about 50 years of cognitive science research, philosophy of mind, etc. Language models are not reasoning engines. Full stop.
https://garymarcus.substack.com/p/llms-dont-do-formal-reasoning-and