You may already have peonies, but a few years ago I moved three to a front bed that runs partially along my twelve foot wide window. These form the foundation for the bed, bloom fairly early here (Mid-June) and they are quite tall, put on a great floral display, and the foliage remains quite lovely throught the season. Between the peonies I have daylilies, lupines, red Maltese Cross, Phlox, and Monarda . These are all tall perennials with very attractive blooms. I fill in with carnations, a carpet rose and annuals. It's still in progress, but getting there.

You didn't mention what kind of exposure your yard gets - is it north or south facing?

If northern or eastern exposure maybe you could try rhododendrons or azaleas as they are evergreens and also bloom in late spring/early summer. Other part shade perennials include hostas, astilbes, ferns, bleeding hearts etc.

If you have a southern or western exposure there are many sun loving perennials you can choose from. Maybe add a columnar and or globe cedar as foundation plantings and intersperce perennials of different heights for interest.

One thing to remember is to space your plants keeping in mind their full height/width at maturity to avoid overcrowding later on. Until they fill in you can plant some annuals to fill in the blank spaces.