As you mentioned it started last week, and now every time I post I get more 'crap' followers. I wish I had answers, but it seems nobody has any idea. I wanted to get the message across, and it seems you have noticed it too.
The reputation system does not have a bearing on the number of followers you have.