No? That's how the TCP 3 way handshake works.

A stateful firewall watches the packets as they come through to track the source port of the initial outbound packet (with SYN flag) to learn the port it should allow the other side to send their traffic back on.

That's why it's terrible if anyone gets control of a computer inside your network: they can now open bidirectional communications, defeating the firewall