Probably Russian olive (Elaeagnus angustifolia L.) or perhaps Autumn olive (Elaeagnus umbellata)? Do a search here on GW or Google them for more info. And if it is one of them, get rid of it ASAP! Cut it down, paint the stump with Roundup. You may have to do that a number of times -- these are highly invasive, non-natives, banned in a number of states (including Connecticut) -- you'll see that silvery foliage along highways everywhere. They destroy native habitat and drive out native plants. There are many wonderful, fragrant native shrubs you could use in its place.

Here is a link that might be useful: Russian olive

If you post a photo, we'll be more likely to help you ID the shrub. Let's hope it's not a member of the Elaeagnus family.

Yeah, probably Elaeagnus umbellata. Makes red berries in the summer that are actually edible, and are higher in lycopene than tomatoes. In the evening the air is perfumed from the Honeysuckle and Elaeagnus that is blooming right now, from the overgrown invasive messes that my neighbors allow to grow. It's intoxicating!

Autumn olive is one of those invasives I feel a bit of regret while cutting down.