Well, I know almost nothing about tomato/tobacco hornworms, but it seems to me that they should start out small.. Check out the link below, there's a pic 2/3 of the way down that shows just-hatched larva that look somewhat like inchworms with stingers... good luck

Here is a link that might be useful: baby hornworm?

You never mentioned what plant you've been finding these caterpillars on. If they are on the tomatoes, then you've probably found some very young hornworms...deebo is correct: they DO start out very small.

There are, however, lots of other kinds of hornworms, so knowing the plant is important.

Hand removal is a very efficient method of controlling caterpillars. You might also look for the small, round eggs. They will be located on the upper or lower surface of the leaves, and deposited singly, rather than in masses like so many other moth species.

I've been finding them mostly on tomato plants -- but I did find a couple on basil and pepper plants--I also found a shiny cloudy white looking egg on a pepper plant which I figured was an egg of these little monsters.

The photo of the very young tomato hornworm totally looked like what I've been finding, and I'm pretty impressed that they grow into such a fancy, big monster.

According to my bug books the larva of just about every memeber of the large Sphinx moth family has a horn. There are a few that do not, but a quick glance indicates the number is not large. Since all of the larva start out from very small eggs laid by the various adults on the host plants they will start out small and then grow, fairly quickly, into very large caterpillars.