Only cut them back if the leaves are yellow & dying, which would be a sign of another problem entirely. I planted stargazers last spring & loved them & unfortunately so did the deer.

The first year that I had mine I cut them back and the following year they did not bloom - in fact I don't think that they even came up! Last year they came up and were beautiful. I just waited a while for the stems to start dying back and then just pushed them over and covered them with mulch. Had a wonderful display again this year too.