ok so the steam deck controller has a thing called "lizard mode" that makes it emulate a mouse and a keyboard

and seems like the only official way to "disable" it is by starting steam and using steam input...