This is true, a workout is fine when it's freezing but if it's already too hot it can be miserable.

Same with trying to sleep. Getting all bundled up and under blankets when it's cold is very comfortable, even if you are still a little chilly. Lying naked on a bed without even a sheet while you sweat is awful.