Depends on how soon you want to eat tomatoes!

Nipping things in the bud is always a good idea if you're not ready, but once those babies get started, well, if the plants are big enough to carry the fruit, why not?

If you have more plants it gives you the opportunity to experiment and see if those early fruits make any difference.

I've got flower buds on three of my plants, but that's all so far.

I have some green tomatoes too. My plants are small (knee high). I'm going let them do what they do and see what happens.