You don't say where you are, but I'll tell you what I did to make mine thrive - plant them in raised beds. I planted my first few butterfly bushes directly in the ground, and they didn't even survive the winter, even though I'm in Zone 8, on the edge of Zone 9, I found out that butterfly bushes HAVE to have good drainage, and our wet winters were what was killing mine.
Since I started planting them in raised beds in my garden, they've all thrived and bloom from spring through fall.

Sherry

Also, they need to be pruned back each spring severely. I take mine back to 18 inches. This year I am late--plan to prune this Friday. We've had too many late freezes. Be sure and not prune until all chance of freezing temps are passed.

And my plants are in raised beds like MissSherry's.

Sandy

Zone 5 is still very cold and having a late spring. I'm in Zone 6 and my buddleia is only about 12 inches tall.

I think you may not be cutting them back. I cut mine down to about 6" every spring. They are much more vigorous when you do that.

Weird hybrids, Ruby Red, also may not do as well as standard varieties. Here are a few tips in the link.

Here is a link that might be useful: Butterfly Bush