Since they bloom on this year's new growth, I would say that you could still prune them but the blooms will be very late.
hortster

Maybe better pruning a little late than too soon. I pruned mine back too early when we were having weather in the 70's back in March and almost killed them. The warm weather went on for so long that I thought it was going to stay that way (it didn't). Anyway, I agree with horster and I'd say to prune yours and have blooms a little bit later than usual, but they will bloom.
Cathy

Thank you!