Hi Kaushik,
I am new to MDM and trying to learn from SCN forums. So bear with me if I am posting something incorrect. 
Are you trying to import(from Excel) in SAP MDM Data Manager or using Import Manager.
Whats the exact version of MDM(Including Service pack) which you are using?
Not sure if below SAP Note will help you
1585178:MDM Import server (MDIS) runs slower than Import Manager
Regards
Arun Sasi
