Looks like the tree is suffering from a common mandarin ailment called alternate bearing. It overloads one year with tiny fruit and skips the next year's production entirely.

Citrus experts claim the best defense is to fertilize extra heavy on heavy bearing years. I have been experimenting with fruit thinning on heavy producing years, and have had some success, on several varieties. Take at least 1/3 of the crop off of a heavy fruit set. Take 1/2 off if you can. This is really hard to do and you have to be able to give your tree "tough love". You may even feel like you're so bad as if you were stomping on baby chickens, pulling off all those tiny fruit, but they will be larger, and juicier, and a lot easier to pick and will look better to boot. I thin twice. First time when they are about 3/8 to 1/2 inch dia. and again at about 1 inch for small mandarins.

As for the fungus, I would take a few leaves to your county (Co-operative) Extension agent, and get a rec. for spray.

Larry

Thanks Larry. The previous owners planted 7 citrus trees in a small area (!!!) and I'm still trying to identify all of the varieties and learn about their ailments. Today I stripped the mandarin tree of most of its fruit (a 10-gallon barrel's worth), as everything was ripe, if small. There's quite a bit up near the tops, still, but I'm having a hard enough time deciding what to do with the fruit I do have.

Do you thin the tree when they're green or just starting to ripen? Or should I have kept more of the fruit on the tree to let it grow to proper size?

Thin at the end of bloom when the fruit are very small. That way the tree can put that growth into producing larger fruit from what is left. As I said earlier, I usually thin twice, since it is often hard to find all the fruit the first time. I have found if I use about the same thinning as on peaches it works on small mandarins. Basically, leave the end most fruit on a branch and thin to about 4 to 6 inches between fruit. Also look at your limbs and be sure you have enough healthy leaves per fruit. This will help give enough sugar to make them tasty.

The very small ones, less than 11/2 in. diameter are hard to juice but if you have that many, it may be your best option.

I have heard people say to prune the blossoms, and I tried that, but they usually just re-bloom. Thinning has worked very well on Page, Kats, and even larger ones like Ponkan and Fortune. The big ones I prune to prevent limb breakage.

What is the active ingredients in your spray? Let's make sure that it will be effective in controlling the scale insects.

The black sooty mold will disappear on its own once the food source (excrement) from the scale insects goes away. In other words, your tree isn't "diseased"....it's infested with scale insects.

One more thing about thinning, if there is a small fruit near the end and the next one up is a little larger, leave the larger fruit.

Another thing with this is limb length. I am 73 years old and grow in raised beds and I don't like the trees getting too leggy which is hard to harvest off ladders on the soft soil.
So I pinch out the bud tips and make them branch-out. This helps a lot in protecting the wood of the tree in a freeze. I go for a compact umbrella shape . That makes for a hardier tree. More limbs every 8-16 inches means too many fruit. Then I am stuck with thinning.

Rhizo is right, I saw the scale too - after my first post.

Thanks so much everyone for all of your advice. I've pretty much done what I can with the existing fruit (10 gallons of fruit, 3 of which is being turned into marmalade, 2 gallons big enough for eating, and 5 gallons into... booze/cakes/??)

So now that I've harvested the tree, should I prune it back or just treat the scale, fertilize, and let it do its thing?