Can you make a raised bed somewhere in your garden with a lot of peat moss and some triple mix soil? Try not to mix in the clay- it will filter in eventually anyway though as your bed is settling down. This is what seems to work for me anyway in my 100% clay base. The south side with afternoon sun doesn't sound good at all either. Azaleas like morning and midday sun, and not too much of it either. They do well in shade as well, even though they don't have so many flowers there for me as the ones which get direct sunlight. Are your azaleas evergreen or deciduous?

The flower buds were formed last summer. If they didn't have buds, they can't bloom. They need some sun to form buds. Too much fertilizer will kill them. They seldom need fertilizer. A good azalea fertilizer like Hollytone can be applied sparingly in the early spring.

The most likely source of death is the clay soil. It drowns them. There is no well-drained clay. Magda's recommendation of a raised bed is ideal for clay soil areas.

Here is a link that might be useful: How to grow azaleas