Because that's why almost no software on the planet does this. It's why HSMs exist in the first place. Private keys must be protected at all costs.

Here's the best attack vector for Matrix:

someone gets your account credentials by one of many different means

they login to Matrix and impersonate you

someone says "hey why aren't you verified? you're not green anymore"

attacker responds "my phone broke, this is my new one sorry I'll approve it when I get home but can you help me with..."

and they'll fucking fall for it. Every time.