ENTROPY and INFORMATION
(see definition of Entropy)

Measure of Information:
Let I(n) the missing Information needed to determine the location of an object whose location is unknown when it is equally likely to be in one of n similar boxes/states.


n = 7

I(n) must satisfy the following:
I(1) = 0
If there is only one box then no information is needed to locate the object.
I(n) > I(m)
The more states/boxes there are to choose from the more information that will be needed to determine the object's location.
I(nm) = I(n) + I(m)
If each of the n boxes is divided into m equal compartments, then there will n.m similar compartments in which the object can be located.

The information needed to find the object is now I(n.m).
n = 4
m = 3

Alternately, the information-needed can be determined by first locating in which box the object is located I(n) plus the information needed to locate the compartment I(m) in which it is located .

These two approaches must give the same information, so that

One function which meets all these conditions is I(n) = k ln(n)
since ln(A.B) = ln(A) + ln(B). Here k is an arbitrary constant.

One Bit of Information : 1 Bit = k ln(2)

1 Bit Information needed locate the object when there are only two equally probable states.

Entropy of a System of N particles in a macrostate j that has Ωj possible microstates accessible to the macrostate,

Sj = Nk ln(Ωj)

Thermodynamic Entropy: k = 1.38x10-23 J/K
Information Theory: k = 1/ln(2) Bits

1 Bit = 9.58x10 -24 J/K
Continued on Free Expansion of an Ideal Gas into Twice the Volume