I used to have a Kingston pendrive. It was mind numbingly slow. You could benchmark the pendrive using HDTach or something and post the results. It could be a Tally specific issue too, since I'm assuming updates to accounting files should be a few KB of data maximum.

If your Kingston drive supports the U3 platform (i.e. a small taskbar icon with launchable icons is created when you plug it in), you could package the tally to run as an U3 application, completely portable. However be sure to backup your stuff properly before you try this!!

Convert and Make Applications Compliant and Run on U3 USB Flash Drive with PackageFactory for U3 My Digital Life