The 'X' comes from the Greek letter Chi, which is the first letter of the Greek word Christós (Greek: Χριστός, translit. Khristós, lit. "anointed, covered in oil"), which became Christ in English. The suffix -mas is from the Latin-derived Old English word for Mass.
There is a common misconception that the word Xmas stems from a secularizing tendency to de-emphasize the religious tradition from Christmas, by "taking the Christ out of Christmas"; nevertheless, the term's usage dates back to the 16th century, and corresponds to Roman Catholic, Eastern Orthodox, Church of England, and Episcopalian[citation needed] liturgical use of various forms of chi-rho monogram. In English, "X" was first used as a scribal abbreviation for "Christ" in 1100; "X'temmas" is attested in 1551, and "Xmas" in 1721.

Your sentiments are well founded indeed and I think it was them jumping on the religious use of the X rather than them introducing the X as a mark to eliminate Christ. Also just another interesting sideline... Saint Andrew was martyred on an X shaped cross so there is plenty of things out there that we can see with eyes of faith that get us around the secular use or meaning!