@haikushack it's not the same people. At least it doesn't seem to be.
Whenever someone follows me I usually check their profile for "undesirable" content. So I would see in my list if someone followed one day and then unfollowed the next.
At least I think I do. I'll try to make an extra effort to see if there's more there then what I'm seeing.
Thanks.