Perhaps snails or slugs. Pour beer in a saucer and place it near the plants and see if you can trap them.

Hope you can find the cause and eliminate them.

Tried some beer last night and all that drowned was a little fly. There were more leaves on the ground, though. I am most likely bringing the plant inside this weekend as we are expecting some cool temps. I hope that the leaf loss stops then!

The tiny scallops at the edge of the missing tissue suggest caterpillar. Can work at night, also from back side of leaf.

Well, I have noticed a squirrel running onto the deck and picking up and chewing on the dropped leaves. Not sure if he is the culprit, but I wouldn't be surprised. We have some incredibly mischievous squirrels here -- they seem to be messing with me just to make me mad. I have yet to see a caterpillar, but some are very good at camouflage and I may not have noticed them.