That photo is post-destruction of course. It was gorgeous before but I don't see a way to post another photo in a follow up.

I'm sad for my plant. Felt slightly bad about killing the caterpillars but I wanted my plant back...

Yes, your plant can recover from this kind of defoliation (destruction is too harsh a word). But I advise you strongly to be more observant of your plant in the future. Google to find out what the eggs look like and destroy them before they hatch.

Neem oil applications can be helpful in deterring the pretty adult moth from laying her eggs.

deercliff, look at the adult moth here;
https://www.google.com/search?q=oleander+moth&hl=en&rls=com.microsoft:en-us:IE-SearchBox&rlz=1I7GGHP_enUS439&prmd=imvns&tbm=isch&tbo=u&source=univ&sa=X&ei=2GqxTpDOLIWItwfu5v2hAg&ved=0CDYQsAQ&biw=1920&bih=898&sei= 3GqxTpChG4ybtwe_16z_AQ
It is highly likely that there are eggs in the planter and pupae in the surroundings. The former will hatch and the very youngest caterpillars cluster under the leaf and you have to look for them. The pupae will become adults and if you don't get those, they will lay eggs in the planter.
Incidentally, how does the oleander caterpillar feed on its preferred host, every part of which is poisonous; and yet the insects escape?
Clue: the oleander caterpillar is Syntomeida epilais

Umm, eggs are on the leaves, not in the planter.

Beyond that, here's info
http://edis.ifas.ufl.edu/pdffiles/in/in13500.pdf

Here is a link that might be useful: info re oleander caterpillar w/images

Info at the Hosta Forum
http://forums.gardenweb.com/forums/load/hosta/msg0513322013993.html?14

Here is a link that might be useful: how to post images

Thank you for all of the help, everyone! I found 3 more caterpillars today and they've been dropped into a bucket of soapy water, as I did with the others.

I've looked for egg clusters on the undersides of the leaves, but do not see any, hmmm....

I wish I had noticed the issue sooner! I usually end up watering in the evenings when it's dusk or dark and I just didn't see the little buggers. I figured the plant wasn't doing well b/c of the sudden cooler temps or not the right amt of water.

Thanks again, I'm clearly not a pro at this but I'm trying. :)