Measure of Information:
Let I(n)
the missing Information needed to determine the location of an object whose location is unknown when it is equally likely to be in one of n similar boxes/states.
![]()
n = 7
I(n) must satisfy the following:
I(1) = 0
If there is only one box then no information is needed to locate the object.
I(n) > I(m)
The more states/boxes there are to choose from the more information that will be needed to determine the object's location.
I(nm) = I(n) + I(m)
If each of the n boxes is divided into m equal compartments, then there will n.m similar compartments in which the object can be located.
| The information needed to find the object is now I(n.m). | ![]() |
|
|
|
m = 3 | |
Alternately, the information-needed can be determined by first locating in which box the object is located I(n) plus the information needed to locate the compartment I(m) in which it is located .
These two approaches must give the same information, so that
I(nm) must equal I(n) + I(m).
One function which meets all these conditions is I(n) = k ln(n)
since ln(A.B) = ln(A) + ln(B). Here k is an arbitrary constant.
One Bit of Information : 1 Bit = k ln(2)
1 Bit
Information needed locate the object when there are only two equally probable states.
Entropy of a System of N particles in a macrostate j that has Ωj possible microstates accessible to the macrostate,
Sj = Nk ln(Ωj)
Thermodynamic Entropy:
k = 1.38x10-23 J/K
Information Theory:
k = 1/ln(2) Bits
| 1 Bit = 9.58x10 -24 J/K |