I have a veerrrry tiny yard, and I'm afraid the ants are going to claim it all in a few years. How concerned should I be about this?

Last year I had a large anthill in my front yard...about two feet diameter. A few smaller hills popped up as well. This year, the hill out front has grown by probably a foot and two more large hills have popped up around the house, in flower beds, etc.

I know there's going to be some ants around no matter what, but is there any way to (non-toxically, I'm hoping) discourage them from starting their own oversized condo developments in my wee yard? And at what point do I need to start worrying about these ants doing permanent damage?

And incidentally, there are lots of birds around...shouldn't they be eating the little buggers?