My guess is that Xavier is just in his head, and doesn't actually exist anymore.