That's the immature seed pods. Depending on what type of peony you have they might be viable, might not, but all have similar pods. The process for growing peonies from seed sounds relatively easy but very long, if you leave the pods on and they are viable you could see baby peony seedlings in a few years without trying at all. There's another post on the forum--Can anyone help me with it? What do I have here? by styxpatron that shows what is probably a result of that inadvertent planting method!

My bush has at least 2 of those seedling pods where there used to be a bloom, about a dozen total. So is it ok to leave them alone or should they be clipped back for winter? How will I know if they're viable?

Do you know the name of your peony? I'm not sure which peony databases have seed viability info, I'd probably start with http://www.paeo.de/name/index.html, they seem to have a lot of info, or start another string with " Is Seed from XXXX Peony Viable?" and see what answers you get. From what I understand most doubles aren't, some semis and single are. There are lots of older threads that talk about gathering/propagating from seed, a search on this forum would probably get some good results. I've never had seed germinate on purpose, I think I might have a few happy accidents growing one of my gardens, though! If you find that your seeds aren't viable I'd go ahead and cut them off just to make the plant look nice again. If they are viable they need to stay on the plant for a while. Good luck!

Energy is going into producing seed that you will probably never use. I find it is best to just remove seed heads to let this energy go back into the plant to benefit next years bloom. Al

This bush will not be damaged by producing a few seeds. Sometimes even a variety known for not producing seeds will produce one or two who can germinate. It would give you an oportunity to produce something unique for yourself, maybe even better than the mother plant. No venture no gain.