More and more of our "wildlife" are adapting to living with us, not out in the woods which are being replaced with housing developments. We are seeing more deer, racoons, skunks, opossums, etc. inside the city limits today then ever because they find food and shelter made by us for them.
About the only way to keep them out of where you do not want them is with good, tight fencing, that also goes undergound at least 1 foot and outward at least 1 foot. A fence just going down to soil level invites them to dig to get to your food source.

About the only way to keep them out of where you do not want them is with good, tight fencing, that also goes undergound at least 1 foot and outward at least 1 foot.

That doesn't always work either. Juvenile groundhogs are excellent climbers. I've been fighting groundhogs for years and I have a strong wire fence that goes 4 feet above ground and a foot below. And they still defeat it in various ways - see the attached link.

They are territorial so if you can kill the ones near your garden you'll at least get a period of relief. When I've been able to find their burrow I've gassed them. You can also live-trap them, but where I live it's illegal to release them off your property so you have to kill them, although you might consider releasing them inside your garden just to see how they are getting in/out so you can plug that weakness.

Here is a link that might be useful: Super Groundhog!