I sympathize--it always looks better "in my mind!" Mind you, we've had an unusual season this year, unusually hot and longer and sooner than usual. Remember last year? Your plants would've looked too short and you would've been lamenting about how nothing's grown in and the garden looks sparse. Same thing happened to me. Thought I found THE plants that worked for my back deck area, only to see it get fried from the sun this year. Last year they did great, but then, it wasn't much of a summer last year.

As for annuals, I've found they don't stay faithful to their descriptions either! Don't despair; I'm sure the gardens at Versailles weren't done in one year.

I know I'm already dreaming of something new to plant for next year!

I know next year I'm not going to plant impatients in full sun, be it the north side of a round bed, in a year with no rain. I'm still askeared of very wide beds.
About 2 weeks ago I visited Spring Ledge Gardens in New London NH. It's really a small nursery, farmstand but they have an absolutely impressive wide bed with some very tall plants that screen the nursery from the street. I walked on both sides just to see everything and ponder how they did it. In some places they do have some low things. I don't get there very often, maybe this was my only visit this year but I thought it looked darn good for mid-August.

BTW I thought my shady garden looked better last year. The little things I planted are bigger but I didn't start the climbing nasturtiums indoors so they aren't climing up the old ladder against the side of the barn like they did last year.

I think you should get rid of the milkweed. OTH my former holding bed that is now a mess of weeds like goldenrod and brown eyed susans is a delight for goldfinches. I kind of hate to cut it back because I can see them from where I sit at the computer.

None of my beds were stunners this year. The constant sun and extreme heat caused some plants to grow ridiculously large, while most of the others had stunted growth. Then the drought fried so many, that there's a possibility they may not be alive next year. Of course this all followed a virtually snowless winter, so coming into the season, we knew there would be issues.

I'll be doing some major work next spring in my once favorite bed. I'm hoping we get a nice rainy spring like the old days. =D