Embed Notice

I have just found an hilariously stupid prompt that breaks the reasoning of every single LLM I've tried it on.

"PHP function that returns true if a variable is null and false if a variable is undefined without throwing an undefined variable warning"

The LLM will always return something along the lines of:return isset($variable) && $variable === null
If you know anything about PHP you would know that this is obviously very incorrect, as isset() always returns false if a variable is null.

The correct answer would be: return ($impactType ?? null) === null
Somehow a LLM never gets this simple question right. So think about that when you ask it to write more complex bug free code.