They're considered non-toxic to humans. Can't vouch for cattle!

http://www.aragriculture.org/horticulture/ornamentals/toxic_plants.htm

Elaeagnus (with two a's) is a large genus that contains about 60 to 70 species including Elaeagnus angustifolia (Russian Olive). It is not poisonous to cattle. Cattle, sheep, and horses will sometimes graze on it, but generally don't care for the taste. Birds, some deer, and elk do use the berries and/or leaves for food.

E. angustifolia is very useful to fix nitrogen in the soil. They have even been planted next to fruit or nut trees as living fertilizer. They aren't nearly as invasive as their relative Elaeagnus umbellata (Autumn-olive).

Thanks for the info also correction in spelling Brandon -- for some reason in Birmingham, they were referred to as "Greek Olive", but I always knew them as Elaeagnus.