Bhagavad Gita, Chapter 5, Verse 21
With the heart unattached to external objects, he realises the joy that is in the Self. With the heart devoted to the meditation of Brahman, he attains undecaying happiness.
Letting go of the constant chase for outside pleasure is what actually makes room for something more lasting.
There's a real trade being described here, not a loss. Loosening the grip on external highs doesn't leave someone with nothing, it opens up access to a kind of steadiness that doesn't run out the way a temporary pleasure does.