Using a tile bag to draw from, definitely increases the wear of your tiles. It wont be much at a time, but after a couple of years of playing, my C1 playset basegame is really showing wear on the tiles.
You can prevent the extra tile wear by putting your tiles in plastic sleeves. These are available for carcassonne tiles, and can keep the tiles safe. Ive recently sleeved my whole C2 playset, as we mainly play that and I almost always use a draw bag