Right, but keep in mind that there is an inherent risk to having a "bad" man interested in you (as in, one who is unwilling to respect your decision if you don't consent to sexual or romantic behavior), so women are more risk averse (on average) in this area than men are, and with good reason since men (on average) are much stronger than women.
So for you touching may be where the line of discomfort is but that may be because you're confident in your ability to put a stop to things before they get out of hand. Now, imagine the woman staring at you was built like a 7 foot linebacker that had poor social skills and didn't understand the word "no", and who had a non-zero chance of forcing herself on you with you not being able to put up the necessary resistance. Perhaps at that point, you would be less comfortable with the staring too, because of the potential implication that it could go further than you want if not nipped in the bud.