there was a jpeg optimization paper that noted about this. they use a geometric mean of like PSNR and SSIM or something. they also added a second pass over the block borders to include those blocky jpeg artifacts in the comparison.

mostly cause PSNR only really evaluates color slopes well, and SSIM evaluates hard edges, and the human brain kind of does a bit of both. although things also get fucky because human eyes don't prioritize each color spectrum uniformly either. tis why a lot of old games had weird shit like 4 bit blues and 3 bit greens