Section 1.1 Degrees of Freedom

Note Id: 202607310001 | Tags: degrees of freedom physics statistics linear algebra machine learning

Across mathematics, physics, statistics, and machine learning, the notion of degrees of freedom captures the same idea: the number of independent configurations available that give rise to a constrained system.

Table 1.1.1. Degrees of Freedom Definitions Across Different Disciplines

Discipline	Definition	Example(s)
Physics	The independent variables required to specify the microscopic state of a physical system.	Position \(q\) and momentum \(p\) of a particle. Normal modes of a vibrating string. Spin states in a quantum system.
Statistics	The number of independent pieces of information remaining after accounting for constraints or estimated parameters that describe an estimator or statistical model.	\(n-1\) degrees of freedom for the sample variance. Residual degrees of freedom in linear regression.
Linear Algebra	Corresponds to the size of the null space (called the nullity) of a vector space.	A column vector \(\mathbf{x}\in\mathbb{R}^n\) has \(n\) degrees of freedom. The rank of a matrix \(\mathbf{A}\in\mathbb{R}^{m\times n}\) has at most \(nm\) degrees of freedom.
Machine Learning	The number of independent ways the model can adapt to a dataset.	Trainable parameters. Effective model complexity after regularization.

Prev Top Next