First, The Roman Empire was always bilingual. Greek always held sway over the eastern half and Latin over the western half. Once the west is gone, of course you'll switch to primarily using the language that most use. The Irish didn't stop being Irish because they primarily speak English now, for example.
Second, the only states that refused to recognize the Eastern Roman Empire as the Roman Empire were the states that were descended from the German invaders and that was because they wanted to claim the Roman mantle for themselves. All of Islam called the East Romans, Rum. As did all of the rest of the world when they had contact with them.
Third, the switch to Christianity had already happened by the time of the split in 395.
Fourth, Rome hadn't been the seat of power for many Emperors since before the Third Century Crisis.
Fifth, states lose and gain territory all the time.
Read The New Roman Empire by Anthony Kaldellis. If you want to read more about this.
Philosophy: about halfway through A Reading of Hegel's Phenomenology of Spirit by Quentin Lauer.
Memoir: last year read Journeys of the Mind: A Life in History by Peter Brown, The Story of a Life by Konstantin Paustofsky.
If we compare with the Roman Republic, not being centered on Rome is a big change. However, many emperors spent little time in Rome, and when there was a civil war it was apparently common to declare some other city a “New Rome,” so that idea was going around. Roman citizenship was considerably more flexible than other ancient empires, including people of many ethnicities, and was eventually extended to every free inhabitant in the empire. They didn’t just call themselves Roman, they legally were citizens, despite not living in and never going to Rome.
We could instead focus on the similarities, like not having a monarchy. Anyone could be chosen emperor if they were in the right place at the right time. (Originally by being a popular and effective general, but later acclaimed by the crowds in Constantinople after political maneuvering.)
So I agree that there’s a “Ship of Theseus” aspect to it, but it seems like a consequence of Roman flexibility, allowing the empire to evolve into something rather different over the centuries.
(Apparently Christianity resulted in far more religious strife, though, compared to polytheism. As a state religion, it seems like it was a turn for the worse, and the history of the many schisms over nonsense is rather tedious.)