Sounds like too much water to me. I've read that many lilies do not actually like full sun, but I have some in full sun that do well, but the ones in morning sun seem to keep their blooms longer. I have Simplton which seems to be one of my later lilies to come up, weaker too than my others, but maybe because the soil is different.
Fridge temps should not hurt them, they need some cold to bloom.

What you have doesn't sound like a sun problem. It sounds like it may be botrytis. Treat with a rose anti-fungal, clean out the top leayer of dirt and any fallen leaves (put in trash not your garden) and make sure when you water not to get any splashing up onto the plant.

-Helen

Search the forum for "botrytis" (and maybe a couple alternate spellings) and a lot of posts will come up.