All I can say is that I've never worried about color differences, and I've never really noticed size differences (except between different varieties). What's your concern? I can't really see anything to worry about other than poorer germination rates, but I've always had plenty good enough germ, and if I were worried about having enough plants germinate then I wouldn't want to discard any ahead of time. If some seeds don't germinate that's no problem for me so long as I have enough that do.