@haayman Thing with Signal is: you do not HAVE to share your contact with the App. And when you do, you can be assured (as long as you trust Signal to have implemented what they have described) that no real phone numbers from your contacts are stored anywhere in their systems.
I decided from the start to NOT share my contacts with Signal and only connect with people I know. A little bit paranoid, I admit, but works GoodEnough for me and my risk calculation. @moehrenfeld