Charles,

From what you are describing and I'm only taking a wild guess that you are dealing with Mealy Bugs.
Are they fuzzy?

Get a Q-tip swab and dip it in alcohol and dabb the bug with it and see if it turns from White to dark in color ( red, brown, etc etc. ).

If there are too many to dabb one at a time then get a spray bottle and fill anywhere from 10% to 25% alcohol and the rest with water and spray the intire tree.
Wipe the pot all around if it is in the pots.

Do this a couple time in a week and they should be gone or put under control.

I have never heard from anyone as for these bugs being harmful but they are feasting on your plants.

Once again, I am only guessing they are Mealy Bugs just by what you are saying.

Pictures will be much more helpful. Could be something else.

Good luck,
Rafed