Embed Notice

@hakona @mhoye
You’ve half got the argument, half missed it.

Yes, as the experiment is set up, experts don’t have much room to overestimate — and beginners don’t have much room to underestimate. Thus even if there is uniform inaccuracy (“noise”) across the whole ability spectrum, beginners will tend to overestimate and experts will tend to underestimate. This is exactly the “it’s just noise” argument, and the whole point of the article linked in the OP.

What you’re missing is that experts do not in fact “get true self-assessment for free,” because they •could• also underestimate themselves — and they do, but (it seems, maybe) by less than beginners overestimate themselves. That conclusion, if it holds under scrutiny, is still an interesting one and not a statistical given.