Shoots without flowers are mainly for gathering energy for general plant health and next year's flowering.

After this past winter's Polar Vortex effects, even the hydrangeas that can bloom on new wood seem to have produced a small number of flowers around the perimeter with lots of flowerless shoots in the center.

If this is outdoors in zone 3, few to none of the stems will survive the winter anyway, and the plant would restart this whole process of new growth from the base next spring.

But, for the sake of plant health, there's no need to cut them down or off until you see what exactly is dead next spring (if no new growth from the stems by June).