While they are not a preferred food source, birds eat the berries sometimes and scatter the seed around the neighborhood. Since I have many trees, they usually scatter them in my garden!
I treat them, as I do Nandina and remove the berries before they fall off and germinate under the plants.
It's your choice.
Rb