You mention clay soil, it could be poor drainage, with water sitting around the roots.

Update: the leaves are getting more yellow, but strangely enough, I am still getting passion flowers and the vines are still growing, but very slowly.

hmm, it could be drainage... but it is planted on an 8" mound...and my bananas in the same planter seem to be doing fine.

Do passionfuits like drier soil?

Did you check the underside leaves for spider mites?

Hmm... I will check. I did have a spider mite problem with the Mandarin.

Since then, I have already sprayed both with Neem, but it has only been 3 weeks. Time will tell if the new foliage is healthier.

Passion fruit usually don't like the neem or any other oil based sprays too much. It can clog the leaves underside and they will die off.