Cutting back drastically on water after the damage has already been done will not improve your plant's chances at recovery. Chronic overwatering causes root rot, and you may have to do some surgery to encourage healthy new roots.

Yellowing can also be caused by other things as well. Underwatering, over or under potting, etc. When was the last time you repotted your plant?

IF you think you've over watered (and you know if you have), then fertilization is the last thing you should do. Remove the plant from the container, prune back the roots to healthy roots, and repot back into fresh (coarse textured) soil.