There is no need to ever remove blossoms from a citrus tree. Most citrus will set way more fruit that they can bear, and hence will drop the fruit at the BB size, and again, at about quarter size. If the tree still has too much fruit at the quarter size, you can pull some off at that time. Pinching off blossoms often just stimulates more blossoms. I think if they are 3 to 4 years old, and have a sufficient leaf canopy, having some fruit is fine. I allowed my in-ground trees to bear, and they are just fine :-)

Patty S.

Great! Thanks Patty, glad I asked, I was about to go pull all of my blooms off of all of my trees.
Evan

I wouldn't do any thinning on that tree, it appears to be quite vigorous. In any case I would wait until the fruitlets are set, a significant proportion of them dropped naturally and only then would I think about thinning out if I thought the fruit would be too much for the branch or I would want larger sized fruit.
Nik

Yes it is growing well, almost 3" of new growth in about 8 or 9 days since I planted the little guy. It's really starting to warm up in the lower desert of California. Thanks all for the advise, I'll leave it alone and let it do what it wants to do.
Evan