Ants aren't really a problem. Although, their presence may indicate an aphid infestation (they like the sap the aphids produce).

For most herbs, aphids, spidermites, and fungus gnats are the biggest pests. You can use a very light dishsoap solution sprayed on, though if the pot is small enough, just tipping it sideways in the sink and washing the leaves can be best. I also use neem on my herbs for aphids, though I still can't keep them fully out of my cutting celery or scented geraniums.

Lauri, have the ants literally taken up residence in the container or are they simply visiting? Frankly, either would not make me happy on my balcony!

I would first try a Terro boric acid ant bait....placed just on the outside of one of the pots. This is a liquid bait that comes in a little clear plastic tray. Once the ants discover the bait, they will swarm the tray by droves. You will be tempted to take drastic action but that is exactly what you want to happen. In a few hours, you will see no more ants.

If they are fire ants and are attempting to colonize within the containers, you'll need to use a granular bait containing Spinosad; It's safe for use on edibles.

Use cinnamon. Ants don't like it. Organic too!