I've gotten those on my baptisias too. They do turn into a moth. I left mine alone but they nearly defoliated the entire plant. It didn't leaf out again so it's looked awful all summer but I do think it's still alive - barely.
Here's a thread from a couple years ago where Lisa posted photos. Butterflymomok identified them as Genista Broom Moth.

Here is a link that might be useful: caterpillars on baptisia