For flowers I suggest

Hyssop. Daylily

Delphinium. Bleeding heart

Foxglove. Virginia bluebell

Lily of the valley. Fire pink

Hosta

For bushes I suggest

Rose of sharon

Button bush

Butterfly bush

Hydrenga

Rhododendron

Azalea. I hope this helps :) Here's a pic of my hummer garden.

Any number of Salvias hardy to your zone.
Annuals such as Mexican Sunflower, Fuschia and Zinnias.
Perennials such as Beardtongue (Penstemon) and Agastache spp.

The one perennial I had last year that they definitely went for was coral honeysuckle. I'm adding more plants this year, but hesitate to recommend anything I haven't had personal experience growing.

Most of my hummingbird garden last year was annuals and tender perennials. If you're willing to try something that probably won't overwinter outdoors, get a Salvia guaranitica. Blue Brazilian sage. They love it.