There is some justification for that belief. On the other hand, Christmas is no more pagan than the cult of Darwinian evolution that is being proselytized by high paid pagan taxpayer supported Government teachers and wealthy pointy headed Government "professors" in expensive taxpayer funded Government indoctrination centers.
Jeremiah 10:2-4: "Thus says the Lord:“Learn not the way of the nations, nor be dismayed at the signs of the heavens because the nations are dismayed at them, for the customs of the peoples are vanity. A tree from the forest is cut down and worked with an axe by the hands of a craftsman. They decorate it with silver and gold; they fasten it with hammer and nails so that it cannot move."