I know you were thinking of perennials, but I am suggesting mostly low growing, drought tolerant shrubs.
I have hardly ever seen blue rug, and some of the other ground cover evergreens fail in full sun and dry soil. If daffodils and other bulbs that like dry Summer soil were placed here and there you would have color for a few weeks. Or you could just have shades of green, greygreen, bluegreen and silvery blue. Check out juniperus 'Blue Star'. I also have the interesting one called 'Pancake' that is the flattest of all. There are untended and bone-dry railroad embankments, locally, that have a Summer cover of Lathyrus latifolia and daylilys. Here in the pine barrens Arctostaphyllos ova-ursi is growing on the sandy roadsides totally ignored and beautiful. Also the plant called sweetfern, which is not green in winter but as tough as nails, two feet tall and naturally spreading.

Uva Ursi is one thing I was considering. Does it require sandy soil? The soil there is loam.

I avoid juniper because I am highly allergic to it (which I learned unfortunately last summer after my hands swelled into sausages and itched for days).

I have only seen bearberry growing in sandy soil, so I cannot tell you how it would do, otherwise. I am sure it does not like wet feet and mine responded very badly to fertilizer of any sort. It is still recovering.
I will e-mail the nursery from which it came, several years ago, and ask if it would do well in loam. I have found them very honest.

Thanks bog turtle. I could easily work some sand into the soil there. I know the drainage is good because it is against the back of our house and the foundation was backfilled with sand and gravel. The location isn't wet because that area has a gable so only normal rain water drips down.

I am finding this west facing rear yard the most difficult area to plan for. It's not "full" sun, and it's all afternoon (lower quality) sun. But shade tolerant stuff gets fried back there because the sun is so intense (no filtration by trees or anything).

My sister also recommended wintergreen, which might work around the corner where is is less sunny.

I don't know if you want an annual cover but there is Lotus vine that will thrive in the hot sun. need a fair amount of water though.
A good Perennial/evergreen is winter creeper. not to be confused with the fruit bearing wintergreen. winter creeper can take a lot of abuse too.