Not sure how to get rid of them though.

In the fall of the year, I have found that the most common insect problems on cannas are grasshoppers (locust), the larva stage of moths (caterpillars) and of course the lesser canna leaf roller. It is likely that if you are seeing a number of wasps, they are feeding on the caterpillars and present no threat to your cannas. By and large the wasp is a beneficial predator.
At this time of the year it is sometimes difficult to control these matured pests. It points to the need of a responsible and effective full season insecticide control program.

Kent

If you have the lily beetles they'll lay their eggs in the soil and you'll have to dig them up and get rid of the grubs or they'll be back next year. They say the only way to get rid of them is to pick them off and drop in soapy water. They do have a bag of some type to catch them with and they can't get out and die. YAY, I'm doing it next year!

Good luck.