“we found no evidence of formal reasoning in language models …. Their behavior is better explained by sophisticated pattern matching—so fragile, in fact, that changing names can alter results by ~10%!”
https://garymarcus.substack.com/p/llms-dont-do-formal-reasoning-and