We are like fish out of water. Just as a fish cannot be happy unless he is in the water, we cannot be happy apart from the spiritual world.
The happiness in serving is the happiness that material nature cannot impede because we can always serve in some way or the other.
When we understand our relationship with God then we all will be happy. It is an essential factor for establishing peace in the world.
Without the grace of the Lord, no one can become happy and prosperous.
Srimad Bhagavatam 3.1.26,