@bbaster @feditips I can't find a way to do it on the official app.
I use Tusky for my app. After I tap on a hashtag and see a list of matching posts, there is a person icon in the upper right with a + next to it. Tapping that will follow the tag, and change the + to X (ie, tap again to unfollow). The website works similarly.