Skip to main content

Section 1.5 Entropy Taxonomy

Note ID: 202604110002 | Tags: thermodynamics, statistical mechanics, information theory
A taxonomy of entropy across various domains.

Subsection 1.5.1 Entropy in Thermodynamics

Definition 1.5.1. Clausius Entropy.

The Clausius entropy is a change in the entropy of a system due to some reversible process where it absorbs some amount of heat \(Q_\text{in}\) at a constant temperature \(T\text{:}\)
\begin{equation*} \Delta S_\text{system} := \int_\text{rev}{\frac{d \, Q_\text{in}}{T}} . \end{equation*}

Subsection 1.5.2 Entropy in Statistical Mechanics

Definition 1.5.2. Boltzmann Entropy.

The Boltzmann entropy of a macroscopic system in a state with multiplicity \(\Omega\) is given by:
\begin{equation*} S_\text{Boltzmann} := k_B \ln{\Omega} . \end{equation*}

Definition 1.5.3. Gibbs Entropy.

The Gibbs entropy of a macroscopic system is defined in terms of the probabilities \(p_i\) of being in microstate \(i\text{:}\)
\begin{equation*} S_\text{Gibbs} := -k_B \sum_i{p_i \ln{p_i}} \end{equation*}

Subsection 1.5.3 Entropy in Information Theory

Definition 1.5.4.

The Shannon entropy of a discrete random variable with possible outcomes \(i\in\{1,2,\ldots,n\}\) where \(n\in\mathbb{N}\) and corresponding probabilities \(\mathbb{p}_i\) is defined as:
\begin{equation*} H_\text{Shannon} := -\sum_{i=1}^n{p_i \log_2{p_i}} . \end{equation*}
πŸ–‡οΈLinked Notes: