Sounds like a watering issue. You need to check the root ball for moisture and not just the soil around the plant. The root ball container mix is typically composed of either peat or bark and if it dries out is hard to get wet again. When you water put the hose right at the base of the plant on a slow trickle and leave it there for 20-30 minutes so that the root ball gets saturated.

OK, I did the slow trickle.
How offten should this be done?
Root ball was damp before planting.
Can this Hydrangea be over watered?

Plant update; About 90% of the leaves are gone now.
Flower heads are there but turning brown quickly.
No new growth is seen.
Looks like this one maybe dead.

Yes it can be over watered. Over saturation of water can cause the symptoms you described. Use your finger to check the moisture of the root ball. If it is moist to the touch - no need to water. Water only when it feels dry. Go ahead and trim the flower heads off. A slight trim should stimulate some new growth.

Don't rip it out. Paniculatas can easily show signs of shock when planted. As long as it makes it to the winter, you'll be fine.

I have worked in a garden nursery industry for over 10 years and every year when the paniculatas start to produce flowers, a number of the leaves on the plants start to turn yellow and fall off. This never happens until they start to flower.....I am convinced that the plant does not have the energy to produce the blooms and hold all of its foliage and so it must defoliate itself at this time.....
That being said, I agree that over watering can produce the yellowing leaves.....but hydrangeas in pots or newly planted out will go through this "yellow leaf stage" until it becomes established......Next year, your plant will drop fewer leaves and the season after that, this "yellowing issue" will not occur once the plant is truly established.

Ian