Yup! Take a hot pepper (jalepeno works fine) cut it up and soak it in a quart or so of water overnight. Strain into a spray bottle and squirt the little devils. the drop dead instantly!
Be very careful though as the pepper spray does burn if you get it into your eyes and can irritate sensitive skin.
Be sure the mark the spray bottle as well :)
I always keep a couple of peppers in the freezer just for this purpose.