It appears that the yellowing leaves are at the tips of the branches. Is that correct? Is there also dieback of stems at the end of the branches? If that is the case, it would be a symptom of overwatering or poor drainage of the planting location.
Yellowing leaves are almost always caused by:
Lack of fertilizer & nutrients
Watering/moisture problems
Insect damage

A balanced fertilizer should be applied in early spring(use of type with micro-nutrients helps). Apply fertilizer again in late summer/ early fall. Many growers use a formulation deficient in Nitrogen(N) at that time, such as 0-20-20, to promote root growth and flower bud development.
You can fertilize them as often as you wish during the summer. A water soluble, balanced formulation is best.
If planted on a slope, drainage is usually sufficient for any type of soil. On level ground, be sure there is proper drainage. Heavy clay will probably need ammendments to accomplish that. You will need to prepare the soil at least twice as deep as the size of the rootball, to avoid creating a natural basin for water retention. Check the percolation rate of the soil in the planting hole by filling with water and check how long it requires to completely drain. A time up to 8-10 hours would be acceptable, but I usually opt for less than 4 hours.
Insect damage. You apparently are well versed on that subject!
Good luck!
Rb

The leaves are turning yellow from the tips moving back into the bush, the ends don't appear to have totally died yet, but that could be next. If the soil is retaining too much water, can this plant be saved? I mean would you dig it up and replant it? I can only think that over the winter this area might have been wet, and perhaps I didn't amend the soil enough when I planted this? So what would be my next step? Pretty soon it is going to need shovel pruning.

I would dig it up and replant. If that is the location you prefer it to be, then dig a hole about 2 ft. deep and wide.
Mix the clay with ground bark and/or gravel, about a 50/50 mix. You could place a layer of crushed stone/large gravel about 6-8 inches deep at the bottom of the planting hole to aid in drainage.
Water to settle the soil around the roots and wait about a week to check for moisture in the root zone. Water only when the top 3-4 inches of soil becomes dry.
Don't add any fertilizer until the plant shows signs of recovery(new growth), which may require some period of time.
Good luck! Hope you are successful, it has very attractive flowers.
Rb