Kate, not knowing where you are makes it harder to answer your question. Are you a warm zone 8 or cool like this one - there are problems that would be more likely where weather is warmer that wouldn't be a concern here.

There is a stem boring insect that can cause branches or entire plants to die, you should be able to see damage in the bark. Root weevils can damage enough roots to kill the plants, but you should be able to see notches along leaf edges where the adults have been feeding above ground.

Do you water or have an automatic system...too little or too much water to those particular plants if they are in the same area?

But often losing plants can be improper installation at planting time. If the roots are not loosened well prior to planting, they can continue to encircle the potting medium until they've exhausted any nutrients available there and literally strangle themselves - often about the third year. Why don't you dig up one of the failing azaleas and examine the roots if nothing else sounds likely to you...tell us what you find.

I'm in Z8, California (San Francisco bay area).

Thanks! I'll try to dig them up and look at the roots...

Morz8 gave you many good ideas. To expand a little, if the roots were not spread out, they can start strangling each other. Root strangulation is best prevented by proper root pruning when planting. If the plant is not too far gone, it might be rescued by digging and removing the soil. Then cutting any circling roots that may be strangling other roots. The roots need to be opened up. On larger plants, some of the top must be removed to compensate for the weak state of the roots. Any time the roots are exposed, they must be kept moistened. Roots that dry out will die. Root strangulation is fairly common with plants that are grown in pots and become root bound.

Phytophthora crown rot or wilt. This root rot is the major killer of rhododendrons and azaleas. It develops when roots are growing in wet conditions. The rot is more prevalent in warm summer conditions. Plants infected with crown rot caused by the fungi Phytophthora have roots which become clogged with brown fungi internally. The roots get blocked and the plant wilts and dies. There is not much of any cure for crown rot. Sphagnum moss and bark dust combined with good drainage seem to prevent crown rot, but do not cure it.

Conversely drought will also create the same symptoms. The plant will start shedding branches, that is they will start turning brown. So it is critical to not over water and cause root rot, but to not let the plants dry out and die.

If the plants are not mulched and the roots get too hot from the sun, the roots can die. The roots are very shallow, so the sun beating down on the soil will heat the roots too much. A 2-3" layer of mulch is ideal.

If you have a ground cover or other plants gowing in the root area of the azalea, it will compete with the very shallow azalea roots and may kill the azalea. Also, if you cultivate around azaleas, you will kill the azalea roots. That is another good reason to mulch azaleas.

Around here we have voles that will eat off the roots below ground level. They are a mouse like creature that loves to eat roots and bark. They cause more damage if the mulch comes right up around the trunk of the azalea. It is best to keep the mulch back about 2" form the trunk.

Entire portions of a plant die if a borer is in a branch. Borers only affect the portion of the plant away from the roots from the borer. If the borer is in the main trunk, then the entire plant will wilt and die. The plant can be save by cutting off the area with the borer and letting the plant regenerate from the roots.

One prolem we find here in the East Coast, if we get plants from the West Coast that are grown in bark dust, when we plant them here we have trouble getting the plants to grow out of the bark dust into the soil. Some times we will dig up a plant and find that it is completely root bound in the bark dust still.

Here is a link that might be useful: How to grow azaleas.