Probably Celandine Poppy or Wood Poppy. Stylophorum diphyllum. The height, 4 petals, time of bloom match.

No, I don't think it's a wood poppy. I checked photos on the internet, and the leaves are different - I should have described them. The leaves on this are long and pointed, a bit wide in the middle.

Evening primrose perhaps?

Here is a link that might be useful: Oenothera

Sure sounds like Oenothera to me, especially with 4 petals. Most likely the species commonly known as "sundrops" (can't remember the species offhand), which is stoloniferous and can spread pretty fast.

Yes, these are sundrops! I took a sample over to the Virginia Garden Extension in my hometown, and they identified them. Wonderfully showy flowers for the garden. Thanks, all.