1. The original references for the Shannon Entropy and the Information theory are
2. C. Shannon, A mathematical theory of communication. Bell Syst. Tech. J. 27, 379–423 (1948)
3. E.T. Jaynes, Information theory and statistical mechanics. Phys. Rev. Series II 106(4), 620–630 (1957)
4. For more details on basic concepts on ensemble theories, see standard textbooks on statistical mechanics in graduate level as exemplified below.
5. M. Kardar, Statistical Physics of Particles (Cambridge University Press, 2007)