No one cares about how you live your life. Do your thing. Don't take anything personally. The good or the bad.
The Art of Not Giving a Fuck says some of the same thing, unfortunately it comes out of the mouth of a born rich kid who never held a real job. He gets upset in his own book about not everyone loving his "advice".
This stuff goes back to Earl Nightingale. If your spending all day worried about what other people think of you, imagining some judgement that doesn't actually exist, you'll accomplish nothing.
We become what we think about. If you spend your free time learning useful skills versus gossiping about people you don't even know, your life is going to be much better.
The Power of Now has an interesting twist on this concept. Well worth the 15$ I spent on the audio book.