Is this a Persian (Bearss) lime? If so, you could harvest them now; but I would wait until they start to get a hint of yellow, when they are at their best.

If this were my tree I would put some Epsom Salt (Magnesium Sulfate) around it; put some on your roses at the same time; can't hurt and usually makes a big difference in the health of the plant. I put Magnesium Sulfate on my lemons twice a year.

I am not sure what type of Lime are they.
They have been on the tree for while now.
Do Magnesium Sulfate increase production and frequency to? I have only 4 on the tree.

Your tree is showing signs of Magnesium deficiency; that is why I recommmend the MgSO4. Age and proper care increase production. From looking at the tree, I am pretty sure it is a Bearss lime; if you don't know better, I would proceed as if it is.

You should be fertilizing your tree at least 3 times per year with a good citrus food, according to label directions. For all my garden citrus I also give them a shot of foliar fertilizer every 15 days; because citrus don't compete well with other garden plants and the foliar gives them essential nutrients that they might otherwise not get. I use Bayer Bayfolan Forte; but others here prefer Foliage Pro; and there are others... they are all good. At maturity, a Bearss lime should produce 2-300 fruits in your climate; here in Guatemala, a good tree will produce 4 times that.

Johnmerr, I will look for MgSo4 around any recommendation regarding mixing/preparation and frequency.
I am having some Gardenia near by would hurt by MgSo4 ?

If you get Epsom Salt, I would put a couple of cups around the drip line of your tree; if you find the real stuff, Magnesium Sulfate, I would put about 1 cup. Don't worry about your Gardenia; it will love it; as will any roses you have.