Bug larvae may compete for food supply with worms, but usually there's plenty of food to go around. The larvae may be able to process certain material that is not yet worm-ready. I don't know if this is true of all larvae poop, but my worms love the poop from BSFL (black soldier fly larvae). Of course larvae eventually hatch, so you may not want to encourage them if your bin is indoors. Not much you can do if your worm bin is outdoors.

Andrew

Cool, thanks for the info. I'm also a bit concerned that eggs and such will end up mixed in with the castings, potentially introducing pests to the plants that I add them to. I guess this is sort of a necessary (small) risk, though?