Ants won't eat your veggies unless some other critter has already taken a bite and left exposed flesh. They go for the sweet substance on the peony buds. I wouldn't worry about it.

You ants are already there doing the job nature entended them to do. Eating things and keeping things tidy. Depending on the type ants do eat the bugs that would harm your veggies. I would have more concern of how wide the peonies could get.