Summer drought or an early frost can cause this, but neither seems likely several years in a row. The other possibility is so-called bud blast, a fungal infection. If it is the latter, buds left attached to the plant will eventually turn fuzzy black as the spores of the fungus form on the bud surfaces. This condition is - maybe - spread by leaf hoppers, so once there's an infection it tends to spread. You need to pick off all the brown buds and any buds that have fallen on the ground before the fungus becomes infectious next summer. There are fungicides recommended for this which need to be sprayed on the newly forming buds next summer. Some recommend spraying for leaf hoppers as well. Your county extension or state agriculture department can give specific advice for your area.

Thanks. I will try a fungicide.