Lily of the Valley is certainly invasive and hard to irradicate if it likes the conditions. It is not attractive all year, like a very few other groundcovers. I have it in two places where it is wanted. If it is not already overwhelming, I would put plastic gloves on and use a washcloth or such soaked in some kind of weedkiller and wipe as many blades as possible with the stuff. The poison should move to the roots. Pretty drastic and labor intensive, but it should work. A neighbor or relative may have their video camera ready and perhaps they will feel their suspicions are now justified.

I'm afraid they are passed overwhelming....if I did try your suggestion, wouldn't the posion spread to the perennials that I don't want to hurt?
My family already thinks I'm obsessed with my gardens and the neighbors aren't close enough to see! I'm ready to try anything.

I do try to get rid of groups of weeds and individual plants among my desired perennials and shrubs. Any overspray is really harmful. I have never noticed ill effects on wanted plants when I get the poison only on the undesirable 'weed'. I wish others would mention any contrary experiences in their gardens. I recently sprayed a truly weedy, ugly flowered little groundcover Veronica between two low growing azaleas, being careful not to overspray. Results are great. You may be at the point of potting up or replacing wanted perennials in order to erradicate the Lily of the Valley. Certainly a lot of work.