Well, I think a good argument is to play the game as intended, then decided if you want to house-rule something. House-ruling really should be left until you've atleast understood a game or you're a game designer looking to understand gaming mechanics. I very much doubt the average gamer is experienced enough to determine a rule is wrong from one or two plays of any game; but you're friends might be special. That's not to say they're out-and-out wrong either: sometimes a mistake or alteration can lead to something better.
Also bear in mind, Carcassonne has had 3 (some would say 4) adjustments to the rules over it's 17 year history. The mechanics have been tweaked to balance play, at least in the Designer's mind.
Here's some game breaking consequences I can think of:
It totally over powers farmers; Just drop them in the largest farm you can, regardless of the tile you got and better yet you can drop them on the best place to glom later on too... only 1-tile away.
Using the Big Meeple from Inns & Cathedrals will be over-powered too, you could just drop it in a vacant city near your opponent, making it much easier to glom on.
The Builder could just be added to any city/road you've already got without having to consider how it's joined on. It's overpowered again.
Does that mean you can drop the pig where you like it to, on your last turn to get some more points?
The ruling totally negates the Dragon from play. Play move, take meeple out of opponent's city. Opponent puts meeple back in city on their next move.
It also makes magic portals useless.
That's my take on the base game and first 3 expansions; heck knows what happens moving on. Obviously your friends will just argue that, the wealth of expansions designed with the actual rules in mind were wrong, so they can just make up their own stuff to work around it. Good luck with that
