Yes, aluminum is toxic to rhododendrons and azaleas. It is used on blueberries, but can kill many other plants.

A wide selection of the native species originating in Asia grow in mountains of dolomite limestone where the pH reading approximates 6.0. The addition of dolomite, which is a combination of magnesium carbonate and calcium carbonate, to our plantings darkens foliage color and increases flower buds. Gypsum, the common name for calcium sulphate, is another type of fertilizer some gardeners seem to have used successfully to improve the quality of their rhododendrons. Epsom salts is magnesium sulfate and it may help.

Is the plant in a lot of shade? It needs 2 to 4 hours of sun each day to set flower buds for next year.

Here is a link that might be useful: How to grow rhododendrons and azaleas.

The plant gets 2 to 3 hrs of sun. How much Epsom Salt for a plant that has a 4 ft spread?

If you buy an agricultural grade of epsom salts, it should have directions on the package. Normal instructions are 1 tablespoon for every 9 square feet. So yours is about 12 sq. feet under the drip line or about 1 tablespoon + 1 teaspoon scattered underneath or dissolved in water and sprinkled underneath.