Mine is planted in the sun, but it is a Pacific Northwest sun, which is not as strong as the California sun. I have found that those buds which are shaded tend to have less (or no) fragrance but are doing fine otherwise. Dropping buds may indicate a lack of water (or perhaps water that's too alkaline).

In your area the best exposure for any camellia is morning sun and afternoon shade. Filtered sunlight either by trees or shade cloth are also excellent exposures. Direct afternoon sunlight in the summer months is a bit too much in your area. Limp or sagging leaves indicate a need for water.