you'd feel different if its the first time you gonna use a Windows Mobile phone. I'd personally prefer SE X1. Its has better build quality, better looks and a better screen. I dont know much about Nokia N97 but have owned the first s60v5 phone by nokia i.e. the 5800xm. I just didnt like the symbian os at all. anyway that was my personal choice. WinMo is a little cluttered but powerful. You have tons of apps out there to download. Assuming you'd prefer the phone for calling, organising, and thirdly multimedia, I'd say go for x1.

P.s. : I own a HTC Tytn II and am gonna sell it to get a x1 or wait for x2 to come in