End-to-end encryption only helps if you are talking to people who you trust. The "ends" — the devices used by people in conversation — really need to be trustworthy. Keep those devices up to date, and take seriously the relationships you're going to include in these conversations.