Just my opinion - but I remove all pots including peat, paper etc. I just peel them off before planting. I have grown a few things in them before (glads and cucumbers come to mind) and they don't break down. In a really wet year or on a plant with aggressive roots they might be okay but in my experience they just make the roots go around in a circle and never decompose.

When I moved here the previous owners had done some landscaping with newspapers underneath bark mulch, and when I ripped it up the newspapers were completely intact even though they were several years old.

I think all newspapers use vegetable inks now, there should be no concerns with safety.

I don't use peat pots frequently but when I do, I find the plants do better if I cut some slits down the corners all the way to the bottom so the roots can escape after they're planted in the ground. It might be helpful for plants planted in newspaper pots also.

I find peat pots often still intact in the ground a year or two later so I guess they don't decompose well in colder environments.

I don't find that peat pots disintegrate here, either. Newspaper does, though. I think one determining factor with the newspaper is moisture. Last year was really rainy, and the newspaper I put down under my mulch didn't make it through the summer.