Could it be that I planted them late and they just havn't started or could I have some other problem?

Yes it could be you planted them late and they just havn't started or you could have some other problem!

Thanks for clearing that up for me. I am sure they will start producing now!

I had problems with my peppers this year also. Early on I had lots of plants whose blossoms kept falling off, they are just now starting to produce half way decently, but still not at all like they should be. I only have a couple of plants with a bunch of peppers on them. I think my problems are weather related. I have been having problems with my tomatoes rotting, and everyone I talk to has also been having similar problems. Just hang in there hopefully you will start to get a bunch of peppers very soon.

Last week I sprayed them with epsom salt and water. It seems to be helping. I might give them another dose later this week. One variety is producing some, but the other has not started yet. Will wait and see.