Bhagavad Gita, Chapter 2, Verse 15

That calm man who is the same in pain and pleasure, whom these cannot disturb, alone is able, O great amongst men, to attain to immortality.

The one who isn't shaken by pleasure or pain is the one who's truly free.

What you're actually after is a kind of happiness that doesn't run out or wear off - and pleasure or pain from the outside world can't give you that. Stop being pulled around by either one, and you're free.