Deer- I have the same problem. What I do is spray the tops (new growth) with a home-made spray. Mix up 2 eggs, 1 table spoon garlic powder, 1 table spoon hot pepper powder, 2 cups water, blend in a elec blender. Add everything to a big jug with a gallon of water and shake up good. Then put some into a hand mister and spray just the fresh little shoots on the top of the plant. Best to let it fermint a few days to let the eggs get stinky. Deer hate this stuff. I have deer around the house constsntly. Spay away! Don't cost you anything so spray as much as you want.
Of course, you could go out and buy this stuff for $25 or $30 bucks.

Best of gardening to you! Dan

I'd agree - deer. Link below is a good pic of what hornworms do. Note they leave the stems.

Here is a link that might be useful:

To answer the second part of your question. Yes- they will return. Once they know where a good snack is, they'll be back. Guaranteed! Maybe one or two days later, but they'll be back. Those fresh little shoots are like candy to a baby. They love them.

This sight is so antiquated, no spellcheck, advanced search, like something from the 80's. JMHO