Is there any way you could post a picture? Sometimes seeing the plant makes it easier to identify the potential problem.

How long has it been since the plant was repotted?

Congrats on getting some blooms this year.

How about Thrips? Those little buggers are notorious for taking out blooms. Loosing the bottom leaves wouldn't concern me too much. Especially if those leaves held out all the way through winter.

Also curious about the size of the plant versus the size of the container and how long its been in its current container.

It sounds like you have worked out a watering and fertilizing system for your locality .Arizona has such a range of climates to deal with I would not be willing to provide a guess on any potential impact of watering/fertilizing. Maybe if you localize to the type of climate (high desert, valley, etc.) someone else can pipe in on it.

this was 4 weeks ago

Personally, I would pull the plant out and inspect the roots and see if it is root-bound, especially if you have had it in that container for more than a year.

When you say 'few new buds', is it forming them and then dropping them before they bloom or just never forming in the first place?

I see one tiny little irregularity on your leaf which can be a strong indicator of thrips present. Even if you don't see them, they could be on your plant (very common). I would give it a spray of insecticidal soap or a mild pesticide.

Like K said, i wouldn't worry over the lower leaves, the top and new leaves are your best indicator. Do they look irregular at all?

Otherwise, to be honest, your plant looks extremely healthy!

Cannot figure out how to flip picture.. But you can see how scraggly the plant looks now with few leaves and brown blooms....I am thinking too little water? Too much sun? too little sun? Not sure what to do since it looked great 3 weeks ago

After looking at your picture I would still recommend checking the plant to see if it is root bound and to spray some mild pesticide. I think it's not enough water over too much water - and if the plant is root bound it is probably not able to get enough water to the roots that it needs. If it had too much water your leaves would be yellow. Those two things are worth checking.

Make sure that white stucco isn't reflecting too much off the plant during the hot parts of the day too.

Its a nice plant that you have. That one bottom leaf that's in the dead center of your last picture looks to me like a lack of water. The plant is putting its energy into the tips, blooms and top leaves which are the newest and therefore the most efficient/effective. Leaving the bottom leaves to dry up. Increase your watering by about half. IOW if you water two times per week add one more day. See what happens. Also give a nice hose down including the bottom of leaves to reduce any thrip or mite population hiding out and keep doing that for about a week. If you can hold off on chemical or oil treatments until the plant recovers a bit I would recommend it.

I think E is right. its in need of more frequent watering and possibly a larger container. Laura's "how to up pot thread" is a great picture tutorial on transplanting and others have posted similar too. I like to use wide squat type pots versus tall pots. Personally I would put a plant that size in at least a 10 gallon container but not everyone wants to be carrying something that size around in the winter. I hope it helps and Good luck with it.

Thank you for all of your comments- I am going to try up watering for a bit and then consider re-potting. I love plumeria and they are SO difficult for me to grow...