It is usually due to mite damage; it also causes the fruit to color prematurely. The fruit is probably usable; you could cut it open to see.

A pictorial survey of citrus ailments at UC --

Here is a link that might be useful: cirtrus fruit problems

This post was edited by jean001a on Thu, Sep 11, 14 at 0:12

jean001a I had found that site with the citrus ailments and couldn't really find one that emulates what I have, which is why I was a bit stumped.

Johnmerr so should I expect the other fruit to develop the same brown area? Most of them are still quite green.

I inspected the leaves very carefully and I didn't see anything that could be a mite, or any bug/pest/mold for that matter. Some of the leaves look like this (picture attached) Not all of them, just a few. I was thinking it was due to too much rain. All the fruit look fine as well.

I'm not meaning to be difficult, but since you said it is "usually" mite damage I'm taking that to mean that it may not be. I'm just trying to save the tree and figure out what this is.

That is the most common cause of such fruit damage. The mite that caused it was there when the fruit was very small; they are not there now, and there is no cure for the damage.
The preventive measures are systemic insecticides and checking the leaves/fruits every week with a 10x loupe, in order to treat with a topical miticide when they are found.
Your fruit damage could be something else; but I don't know of anything other than mites that cause that. The additional evidence is the fruit is prematurely colored... normal would be November/December.

Well I guess I'll pull out the neem oil and hope to head off any future problems. Thanks Johnmerr, it's appreciated.

I have a big lemon tree and I get the same thing here and there even on high branches. I can't find what it is either. It's only on the bottom as if the juice sinks to the bottom and turns brown.