Even one of my favorites, Isla Mujeres (which I still love!) is getting too busy and full of annoyingly bad and overpriced touristy restaurants. Holbox is about to fill that slot. Holbox does indeed cater to tourists so you have a lot of fun bars and restaurants, but it doesn't have that "tourist crazy hype feel" ... yet.
Holbox, also famous for swimming with whale sharks (amazingly cool if you haven't done this already!), has the clear turquoise water, lazy sandy streets, chilled out feel you've been looking for. Go on a budget or fancy it up in a nice resort, you'll quickly be thinking like Brigham Young "this is the place". Oops, sorry for the Mormon reference, I used to live in Park City. ;)
It's a bit harder to get there than say, to Isla Mujeres or Playa del Carmen, but it's pretty easy in the whole scheme of things. Take the ADO bus about 3+ hours North from Cancun, then a ferry out of Chiquila, a golf-cart taxi into town, and you're in your new paradise. Very few cars, lots of hammocks and cold drinks.