Independent agencies are often a "good idea" but it's not the case that it's a good idea forever, and they are still a part of the executive branch just independent by legislation, Congress can let Trump do this.

This won't affect your rights, it might affect your mail.