Reddit mentions of Complexity, Entropy and the Physics of Information

if you really want to understand entropy, don't bother studying physics. information theory will give you a much better understanding -- i'd recommend Cover & Thomas's book, really just the first few chapters, up to the stuff on high-probability sets and the typical set. there's very little intuition to be gained from the "concrete" definitions of entropy in physics.

and if anyone says that entropy is a measure of "disorder", ask them to define order. if they say something about the number of "microstates", ask them why they chose a particular course-graining (that definition only holds for discrete systems, not a system with an energy dependence on a continuous parameter).

Zurek, in his foreward to complexity, entropy, and the physics of information, states:
> Since the introduction of Maxwell's Demon and, particularly, since the celebrated paper of Szilard and even earlier discussions of Smoluchowski, the operational equivalence of the gain of information and the decrease of entropy has been widely appreciated. Yet, the notion that a subjective quantity such as information could influence "objective" thermodynamic properties of the system remains highly controversial.
> It is, however, difficult to deny that the process of information gain can be directly tied to the ability to extract useful work. Thus, questions concerning thermodynamics, the second law, and the arrow of time have become intertwined with a half-century-old puzzle, that of the problem of measurement in quantum physics.

in the twenty years since that was published, much work has been done and an operational definition of entropy as the kolmogorov-chaitin complexity of a system seems to be fairly well accepted among those who actually care about having physics on a firm basis. only trouble is that k-c complexity is not computable...