The entropy is a measure of uncertainty that plays a central role in information theory. When the distribution of the data is unknown, an estimate of the entropy needs to be obtained from the data sample itself. A semi-parametric estimate is proposed based on a mixture model approximation of the distribution of interest. A Gaussian mixture model is used to illustrate the accuracy and versatility of the proposal, although the estimate can rely on any type of mixture. Performance of the proposed approach is assessed through a series of simulation studies. Two real-life data examples are also provided to illustrate its use. (C) 2022 Elsevier B.V. All rights reserved.

Mixture-based estimation of entropy

Scrucca, L
2023

Abstract

The entropy is a measure of uncertainty that plays a central role in information theory. When the distribution of the data is unknown, an estimate of the entropy needs to be obtained from the data sample itself. A semi-parametric estimate is proposed based on a mixture model approximation of the distribution of interest. A Gaussian mixture model is used to illustrate the accuracy and versatility of the proposal, although the estimate can rely on any type of mixture. Performance of the proposed approach is assessed through a series of simulation studies. Two real-life data examples are also provided to illustrate its use. (C) 2022 Elsevier B.V. All rights reserved.
2023
File in questo prodotto:
Non ci sono file associati a questo prodotto.

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11391/1552454
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 4
  • ???jsp.display-item.citation.isi??? 4
social impact