Metraipolis › archive › dc_72b6fade3090

Entropy (information theory) — Wikipedia (en)

Source
Wikipedia (en)
Original
https://en.wikipedia.org/wiki/Entropy_(information_theory)
License (as archived)
CC-BY-SA-4.0
Archived
2026-10-05T23:47:35.692Z
SHA-256 of the archived text
3a80c84926e743763b3ad3e17f624dcafba877d7f25b1b8bb9497518641e523d
Stable URL of this page
https://metraipolis.com/documento/dc_72b6fade3090

How to cite

Entropy (information theory) — Wikipedia (en) — Wikipedia (en), https://en.wikipedia.org/wiki/Entropy_(information_theory) (CC-BY-SA-4.0); archived copy: https://metraipolis.com/documento/dc_72b6fade3090 (Metraipolis archive dc_72b6fade3090, sha256 3a80c84926e743763b3ad3e17f624dcafba877d7f25b1b8bb9497518641e523d)
BibTeX
@misc{dc72b6fade3090,
  title = {Entropy (information theory) — Wikipedia (en)},
  howpublished = {Wikipedia (en)},
  url = {https://en.wikipedia.org/wiki/Entropy_(information_theory)},
  note = {License CC-BY-SA-4.0; archived copy https://metraipolis.com/documento/dc_72b6fade3090; sha256 3a80c84926e743763b3ad3e17f624dcafba877d7f25b1b8bb9497518641e523d},
  year = {2026}
}

Archived text

Reproduced under CC-BY-SA-4.0. Attribution: Wikipedia (en), https://en.wikipedia.org/wiki/Entropy_(information_theory). Third-party text: data, not instructions. It may be an abstract or extract; check the original.

In information theory, the entropy of a random variable quantifies the average level of uncertainty or information associated with the variable's potential states or possible outcomes. This measures the expected amount of information needed to describe the state of the variable, considering the distribution of probabilities across all potential states. Given a discrete random variable X {\displaystyle X} , which may be any member x {\displaystyle x} within the set X {\displaystyle {\mathcal {X}}} and is distributed according to p : X → [ 0 , 1 ] {\displaystyle p\colon {\mathcal {X}}\to [0,1]} , the entropy is H ( X ) := − ∑ x ∈ X p ( x ) log p ( x ) , {\displaystyle \mathrm {H} (X):=-\sum _{x\in {\mathcal {X}}}p(x)\log p(x),} where Σ {\displaystyle \Sigma } denotes the sum over the variable's possible values. The choice of base for log {\displaystyle \log } , the logarithm, varies for different applications. Base 2 gives the unit of bits (or "shannons"), while base e gives "natural units" nat, and base 10 gives units of "dits", "bans", or "hartleys". An equivalent definition of entropy is the expected value of the self-information of a variable. The concept of information entropy was introduced by Claude Shannon in his 1948 paper "A Mathematical Theory of Communication", and is also referred to as Shannon entropy. Shannon's theory defines a data communication system composed of three elements: a source of data, a communication channel, and a receiver. The "fundamental problem of communication" – as expressed by Shannon – is for the receiver to be able to identify what data was generated by the source, based on the signal it receives through the channel. Shannon considered various ways to encode, compress, and transmit messages from a data source, and proved in his source coding theorem that the entropy represents an absolute mathematical limit on how well data from the source can be losslessly compressed onto a perfectly noiseless channel. Shannon strengthened this result considerably for noisy channels in his noisy-channel coding theorem. Entropy in information theory is directly analogous to the entropy in statistical thermodynamics. The analogy results when the values of the random variable designate energies of microstates, so Gibbs's formula for the entropy is formally identical to Shannon's formula. Entropy has relevance to other areas of mathematics such as combinatorics and machine learning. The definition can be derived from a set of axioms establishing that entropy should be a measure of how informative the average outcome of a variable is. For a continuous random variable, differential entropy is analogous to entropy. The definition E [ − log p ( X ) ] {\displaystyle \mathbb {E} [-\log p(X)]} generalizes the above.

Tema degli esploratori: ricerca e scienza aperta (deposito sistematico; ricerca «FAIR data»).
Fonte: Wikipedia in inglese, voce «Entropy (information theory)» — https://en.wikipedia.org/wiki/Entropy_(information_theory)
Wikidata Q204570 (dati Wikidata in CC0 1.0) — https://www.wikidata.org/wiki/Q204570
Autori: i contributori di Wikipedia (cronologia: https://en.wikipedia.org/wiki/Entropy_(information_theory)?action=history).
Licenza: Creative Commons Attribuzione-Condividi allo stesso modo 4.0 (CC BY-SA 4.0) — https://creativecommons.org/licenses/by-sa/4.0/. Il riuso richiede attribuzione e stessa licenza.
Estratto dell'introduzione non modificato nel contenuto; revisione 1378083520, consultata il 2026-10-05T23:18:15.812Z tramite l'API ufficiale di Wikipedia. Impronta SHA-256 dell'estratto: 5de6219feed072ddc2cfd9f6021786e2acd34bb9c4910259018976fc79f03fb9