"Major insect pests of peppers in Texas are leafminers, cutworms, aphids, spider mites, flea beetles, pepper weevils and fruit worms. Pepper weevils are especially devastating to jalapeno peppers....." This is from an Aggie Hort article that goes on to explain when the weevil punctures the pepper, the pepper may fall from the plant. Are you finding them on the ground?
From your description, you might suspect the weevil. Have you gone out at night after dark with a flashlight and looked?

Here is a link that might be useful:

By the way, insecticidal soap won't act as a 'deterrent'. Neem oil might do that job for you....it's an anti-feedant.

That's interesting that they'd go for the seeds. I haven't heard of any pests doing that. But otherwise, it sounds slightly similar to things that leaffooted bug nymphs do down here in southern Texas with tomatoes.

So you don't see anything during the day that looks suspicious? Then a night creature might be involved, which changes the culprits. Yeah, as Morz8 mentions, go out at night with a flashlight to investigate. Good point too rhizo_1...