Just to be on safe side, maybe you should cover them. The hard freeze won't kill the plants, but they may damage/kill the flower buds. In that case, you won't have any flowers for this season and that would be a shame.

Yes cover them up. Even if the plant root survives, you'll dread seeing the dead leaves/stalks for a few weeks anyway. Compare that to opening up your cover tomorrow morning and see the weather temps back up and your plant doing just fine because you took a little bit of extra precaution.

I also learned my lesson this year about certain other plants (not peonies). Their roots may do fine with the low temperatures but their leaves and stalks that are up above ground need to be protected (even a new leaf compared to an old leaf, the new leaf needs more protection on the same plant)

If they have flower buds cover them (a gallon ziplock baggie works nicely). If no flower buds, don't worry about it. They are cold weather flowers.

The most damage that can occur for your plants is if the temps goes from 24 to say 60 degrees in a 24 hours period. It is the sudden freeze and thaw that the leaves and buds can't take. However, if the freeze and subsequent warmups are gradual, then the damage maybe lessened.