Are they fire ants?

Ants will invade plant pots but will seldom nest there unless the soil in the pot is too dry, because ants do not nest in wet soils. If they are simply there and not nesting they may be there because there is a food source, maybe an insect that you may not want in that pot. As a rule, ants are not the problem we have been raised to believe they are and they are more beneficial then harmful.
However, the Imported Fire Ant is a problem and where they are people need to do what they can to control them.

Here is a link that might be useful: Imported Fire Ants