It may be best to go ahead and pick any fruits that have started to blush.

I pick all the fruit at the first sign of late blight. If you wait, even the fruit that look unaffected will turn to brown mush very soon after the plant starts to blight, whether you pick them or not. On my big beef plant, just the top few branches were blighted. I picked 80 fruit that appeared unblighted. Only half of those were unaffected by blight and the good ones all ripened fine eventually. I waited a little longer with some plants and almost every fruit became infected and had to be tossed. Late blight is a real scourge around here.