Reddit mentions of Farewell To Entropy, A: Statistical Thermodynamics Based On Information

Here is a good book on this subject.

The argument therein is that entropy in both cases quantifies "missing information." Given a set of prior information, entropy tells us how much more information is necessary to specify the exact state of a system.

In thermodynamics, the prior information is the thermodynamic variables (temperature, pressure, etc), and the entropy tells us how much more information is necessary to specify the microstate.

Finally a question I can answer!

Entropy is a highly misunderstood concept and even many scientists struggle with misconceptions. Also, it can get into lots of subtle details when you start getting into the nitty gritty of it, and this can get into research-level questions, even in 2012. People have written books about the proper way to calculate it: http://www.amazon.com/Farewell-To-Entropy-Arieh-Ben-Naim/dp/9812707077

You have stumbled upon a subtle fact that many overlook: indeed it is the case that in the purely classical gas (particles can have any real-valued position and velocity), absolute thermodynamical entropy, in the information-theoretic view, is infinite.

You can still use the 'number of microstates' definition to arrive at a finite quantity, but you get something more similar to this: http://en.wikipedia.org/wiki/Differential_entropy

The basic idea is that you're calculating not the absolute entropy but the entropy relative to your units of measurement. The unit of entropy in SI is J/K, and the fact that you can have an infinite number of states for your particles is 'cancelled out' by the fact that your units can also take any real value. I hope this helps.