My pet peeve: when the core of a challenge in a game is something closely tied to the player's physiological limits, such as reaction time. The challenges get progressively harder, and at some point they get close to or even past the player's hard limit, and at that point the player can no longer continue. Happens to me a lot in some more dexterity-based games (e.g. in Hollow Knight, which i never finished for this reason).