All holsters will cause wear on a guns finish. That is true of leather holsters and it is true of Kydex. Kydex may wear the gun a little faster. But if you are going to put a gun in a holster (any holster), you will see wear on the gun.
I understand that, I am really impressed with how my Kimber has held up to the Galco crossdraw I have had it in for 5 years. I dont wear it everday, just while at the farm or when hunting, but it has shown remarkable wear resistance.
The biggest cause of holster wear is a "dirty" holster. While holster wear is a simple fact of life, a clean holster will wear on a gun much less than a dirty one. If your holster is leather check with its maker to see what they recommend for cleaning. If kydex a damp cloth and a little liquid soap will do the trick just fine - just make sure the holster is well rinsed and dry before you reinsert your gun.
I've always found the kydex holsters to wear the finish less than the leather holsters, and usually only in specific spots. The leather holsters tend to wear more of the finish as they tend to contact more of it.
You also really dont have the rust issue with kydex like you do with leather.