The caterpillar that you have illustrated is the caterpillar of the Black Swallowtail butterfly. These caterpillars will be found on parsley and very closely related plants and not on apples, etc. I suspect that this is a case of mistaken identity.

On their host plants (parsley, carrots, dill, etc.), these beautiful caterpillars will gladly eat until there is no more, then will find a place to pupate so that the transformation into a butterfly can occur.

Oh boy. So, how can I get rid of them without hurting my plants?

Because they're found on food plants such as parsley and dill, the only solution is to hand-pick them - but then you won't get the butterflies they are soon to become. You can't poison them because then you can't eat your herbs.

Dill is so easy, I just plant extra and let them have it. Dill, fennel, parsley, rue - I see someone mentioned carrots but I have no experience with that. If I really wanted to save my dill, I'd pick them up and put them on the fennel, which grows like a great bronze cloud with no help from me.

They taught me to respect my rue. They ate it down to nothing in practically no time - just a tiny green stub - and it came back!