Also, some other red peppers have these empty capsules on them. It looks like abandoned egg sacs of some sort (picture 2). Any idea what I'm dealing with?

The lack of a living pest inside made me think Pepper Maggot, eggs are inserted and the larvae eats it way out BUT, the chewing on yours and mine indicate something ate its way in.

I've also had issues with earwigs punching holes in peppers and moving in but the holes are larger.

I had the same problem in 2010:

Unfortunately for me, odd are it wasn't weevil. Up north here they're only a problem if they can over winter in a greenhouse. As far as I can tell, there ain't no greenhouses within bug flying distance to me.

An excerpt from an recent Ontario agriculture report:

This pest does not overwinter in Ontario fields (it requires living hosts year round to survive), so its presence would be of concern to field pepper growers only if it escaped a greenhouse/packing shed or was present on waste plant material transported from these operations during the growing season.

The adult lays eggs in the wall of the pepper fruit. The larva emerges and moves directly into the fruit, so it is difficult to detect and is protected from insecticide applications. Eggs may also be laid on flowers and buds. Adults feed on fruit and flowers, but prior to flowering, they will feed on stems and leaves. The most serious concerns are for the presence of larvae in the fruit and premature fruit drop due to adult and larval activity. The pest could complete several generations through the growing season.

Host plants include peppers, nightshade, and eggplant. It will also feed on tomato, petunia, horsenettle, and other solanaceous weeds.

If it is the pepper weevil, how do I control them?

I understand your desire to quickly rid yourself of the pest to save your crop but it's best to id the pest before you start applying fixes. By applying the most effective fix for your pest you'll save time, money, effort and most importantly not kill off any beneficial insects a scatter shot might.

Insect strips can help you trap and identify pests on your plants. I use Safer's sticky sticks (pic below) but you can fashion your own traps using fly paper.

I believe greenman28 once said nuke em from orbit" when discussing the purely evil spider mite.

I have however suggested in the past resort to full chemical warfare only when you're going to loose the plants anyway. You'll still probably loose the plants but you get some satisfaction out of unleashing the wrath of Dupont or Dow on the little buggers

Anyhow, I still contend it's best to know the enemy you're dealing with... a 303 is overkill on a squirrel but a pellet gun on a bear will get will get you killed ;-))

Here's another pic of the damage if this helps.

I would put some of them low and some higher up around a couple of plants. The sticky sticks I use come with stack-able posts so you can increase the height. You could fashion your own with varying lengths of sticks etc. and fly paper.

Unless you lay some on your soil, sticky traps may or may not catch your pest if it isn't a flying one. Also, the traps are indiscriminate, they'll catch both good and bad so you'll need to do some detective work.

You folks in the US have a wonderful resource with your USDA Extension system. Have you thought about checking with your Extension? Based on their regional experience they may be able to help you narrow your problem down or even offer a solution.

Just a thought.