If you found it growing in the wild, there is a pretty good chance that it is something non-native and invasive. Autumn Olive (Elaeagnus umbellata) fits the description.

I'm afraid it is autumn olive. I had also considered this, but the leaves didn't look quite right in the pictures I had looked at. After reading sam_md's post, I looked at some more pictures, and the berries are a perfect match, right down to the little brownish spots on them. I checked while I was out walking today, and the undersides of the leaves are indeed silver.

Mystery solved, I guess. Too bad. It's really a very handsome plant. I've always read it was a very invasive, undesirable plant, but I can see why it gained popularity when its planting was encouraged. Unfortunately, this particular shrub is located on private property, so I won't be able to chop it down. At least now I'll recognize it if it shows up on my property. I can add it to my list of things to eradicate, along with oriental bittersweet, mile-a-minute vine, garlic mustard, . . .
The list never ends.

Thanks to both of you for your help.

If it's on 'waste ground' or land about to be developed, I'd be inclined to cause it to 'happen to have an accident' if possible, before the birds get all the berries.