http://www.empireonline.co.uk/site/news/NewsStory.asp?news_id=16892
Must be something to do with the high-life!
Not a shock really as Mr Cumming disliked the previous X-Men director Bryan Singer:
Alan Cumming surprised journalists when he answered the simple question of, "Are you disappointed Bryan Singer won't be directing X-Men 3?" with a resounding, "No." He would not give details about the feud, but made it clear he's looking forward to a new boss on the third film.
"I'm not disappointed, I can't deny it," Cumming said. "I think he's really talented. I'm very proud of the film. I think it's a great film. I didn't enjoy working with him on the film."
http://www.latinoreview.com/interviews/cumming-x-men.html
So if you badmouth the former director, you're not asked back!
I never liked Nightcrawler in the film anyway.
And even in the comics I liked the stories that played up the swashbuckling angle and played down the religious indoctrination of his upbringing.