I played the original Carcassone before playing Carscassone the City. Both games are great, but they're not really different enough for most people to bother owning both.

Both games have the advantage of quick, easy setup, since the game is building the board. They both work well with two players (or more). They've also got a near perfect balance of strategy and luck. Enough strategy to make the game engaging every time you play but enough luck to prevent cut-throat competition and hurt feelings (most of the time). They're also both easy to learn; one game was enough for me to understand the rules of either (though the certainly sound complicated at first)

So on to the differences:
-Something about the farms in Carcassone confuses me; it just takes so much effort to figure out where the borders are. The markets in Carcassone the City serve a similar role but are much easier to understand.
-Carcassone the City also has the fancy wooden box if you care about those things. Personally, I prefer the cardboard box that seems to be designed for the pieces to the big open box.
-Carcassone the City has some interesting twists/variations on Carcassone that make the game more interesting and fun.
-However, Carcassone the City does not work with the expansions to Carcassone. You're limited to 4 players and the current pieces and rules.

If you ever want to play with more than 4 people, get Carcassone and an expansion set (I recommend Traders and Builders). If you're okay with the four person limitation and no expansions, Carcassone the City is definitely more interesting and fun than Carcassone without expansions.