It depends on your requirements. Defy overall is a better model hardware wise despite having only 800 mhz. But each model has its own advantages.
I have personally used Xperia X10 for a week and was satisfied with it. Do not expect it to be a gaming machine as Adrenos 200 gpu has its weakness. The most I liked about Sony X10 was its 8 MP camera/video recording, excellent processor that is fast for functions other than gaming, Excellent form factor and also good UI. But if gaming is part of your regular usage, you might dislike Xperia X10 instantly due to Adrenos 200 gpu. It does play older games with ease, Just not the newer ones that are becoming more 3d intensive. Even though RAM is only 384mb, it breezes through regular functions with ease. I was still able to play some games with ease like Asphalt 5, Hero of Sparta and Crusade of Destiny.
I have never used Defy, but had a chance to hold it and test it. It is rugged and tougher build as all the other members have said. It does last you longer than Xperia X10 despite heavy usage. Also the custom mod is available for this phone that gives you option to overclock it or tweak its performance for getting maximum output. But keep in mind, that even though Defy's gpu is better than Adrenos 200, but not by much. So do not expect that Defy will have huge gaming performance gains compared to Xperia X10 It uses Powervr sgx 530 which is inferior to Apple Iphone 4's GPU or Galaxy S GPU or even Adrenos 205. Other performances is equal to Xperia X10.
CPU processing wise 1st generation Qualcomm snapdragon QSD 8250 (one that X10 has) is much better than Hummingbird or TI. Only in GPU qualcomm has its weakness. So for general purposes I would choose Qualcomm. If you are interested in gaming, why not choose hummingbird or even second generation Qualcomm processor that comes packed with Adrenos 205, which is much better than Defy's GPU.
Sorry for long post. In conclusion if you decide with either, you will not go wrong. It just depends on your requirements. Like said before, If gaming is part of your regular usage, then look for phones with hummingbird processor coupled with sgx540 or second generation qualcomm scorpion coupled with Adrenos 205.