Yes, it is a problem and I'm not yet sure what steps to take also. Look for a pdf on this link page, which is a research review of urban contaminated soil and instructions for sampling. The local ag extension service may provide testing kits -- they did in the past, but I haven't checked my garden for over a decade. I had always assumed it would remain safe, but then I started to wonder about it this season.

When I contacted the NY extension service years back, they were quite flustered by someone claiming to be an urban gardener -- eventually I convinced them there were bits of real land here in Manhattan. They've gained some experience since then.

I have heard of gardeners removing the top 6 to 10 inches of soil and replacing it with growing composite. And the person writing the blog in the link promotes a type of containered plantings -- perhaps plunge them into the existing beds.

Let me know what you do. I'm only growing basil and greens, and not as much as I'd like because a huge new building is now blocking much of my sunlight.

Here is a link that might be useful: insideurbangreen

My advice would be to completely remove all the soil because it has probably broken down into such small particles that it will be hard for roots to get any oxygen. I would then fill the whole thing up with turface and grow it as though it is a "to waste" hydroponic system. That is basically what places like disneyland do with there long term planters when they don't want to have to replace the potting medium ever, except they use very very coarse (like greater than 1/16" grain size) sand.

Also, the main source of lead in urban soils is from house paint. Over the years as lead based paint was scraped away, it would fall to the ground and not be cleaned up. Decades of this led in some places to serious contamination. Lead in gasoline caused contamination but not as concentrated as the hose paint. You probably don't have to worry too much about either of those factors in your planters.