I've never had that happen. This year I put out way fewer nesting sites than I knew they would need. After everything was full, there was a week or two of continued activity around the full, plugged nesting tubes. They were desperately searching for more places. I could literally hold a tube up in the air and it would be moved into within seconds. Yet none of the exiting mud plugs have been tampered with, by bird or by bee.

Excellent response. Thank you. I really did suspect woodpeckers as we have numerous Hairy, Downy and a couple Sapsucker that hang around the yard and could be the culprit(s). Just before initially posting this, I had a Hairy on the outside eaves of the house attacking a small paper wasp nest eating the adults and larva.
My first couple seasons with BOBs is as you described, and only the last 2 seasons have I seen this damage occur. I've got new plans in place for next year that should solve the issue.
Thanks again.