In the leaf that goes off to the left, you can see the caterpillar (or perhaps the larva of a different type of creature) curled up, half under the small yellow leaf which he's trying to hide behind. If you can't see him, he's the light gray half-circle.

I don't know what kind of worm he is -- maybe some kind of leaf-curler, as he seems to be adding web to the area where he's hanging out -- but it looks like he just nibbles the lower layers of leaf. I doubt he's the cause of the older damage, the parts of leaf missing along the edges.

For the yellow leaves and other issues, it would help if you could post photos of those areas. [Post them in this same thread. If the software gives you problems, you may need to change the text in the Subject of Posting box.]

Scot, I see some old Citrus Leafminer damage, which can be treated next season (search our forum, many posts about how to tree this pesky but rarely fatal pest). The chewing could be from many different insects - hard to tell you because we don't know where you live. Could be snails, slugs, grasshoppers, the larvae/caterpiller of the Giant Swallowtail butterfly to name a few of the more common offenders. But, what I'm more concerned about is the overall health of your tree I think I can see in the background. Are you fertilizing your tree regularly? If so, with what and how? How are you watering? Is the soil too wet or too dry? You don't say what location you're in? Can you provide more photos of your tree overall?

Patty S.