hornworm larvae generally come from the soil and will go after almost every plant(peppers and tomatoes are generally first)

The best way to get ride of them, is kill them as you see them, you could spray but it'll take multiple sprayings

also if you see one that looks like it has white worms on it, let it live those "worms" are actually parasitic wasps

Hornworm eggs are laid singly on top of the leaves. They are small and milky white. Dust the plant with a product containing BT like Dipel or thuricide. If you see any worms with the white cocoons, leave them alone to spread the tiny parasitic warps. While you're at it dust any other solanums you have, like tomatoes, eggplants or other peppers. Repeat after a rain.

I forgot to say BT can be used up until the day of harvest, just wash it off and eat.

there is a product at most garden centers called BT worm killer by green light it is a liquid that you put like 1 teaspoon per quart of a spray bottle and spray your plants and ground at bottom of plant about every 5 days or so and after rains. when a worm bites a leaf, he will die the next morning. I had a huge infestation on tomatoes and after spraying a couple times they disappeared. no more holes in my tomatoes.

there is a product at most garden centers called BT worm killer by green light it is a liquid that you put like 1 teaspoon per quart of a spray bottle and spray your plants and ground at bottom of plant about every 5 days or so and after rains. when a worm bites a leaf, he will die the next morning. I had a huge infestation on tomatoes and after spraying a couple times they disappeared. no more holes in my tomatoes.

I used BT earlier in the season and have no hornworms at all this year on tomatoes or peppers.