Advancing Shannon entropy for measuring diversity in systems

Rajeev Rajaram, Brian Castellani, A. N. Wilson

    Research output: Contribution to journalArticle

    2 Citations (Scopus)
    50 Downloads (Pure)

    Abstract

    From economic inequality and species diversity to power laws and the analysis of multiple trends and trajectories, diversity within systems is a major issue for science. Part of the challenge is measuring it. Shannon entropy H has been used to re-think diversity within probability distributions, based on the notion of information. However, there are two major limitations to Shannon's approach. First, it cannot be used to compare diversity distributions that have different levels of scale. Second, it cannot be used to compare parts of diversity distributions to the whole. To address these limitations, we introduce a re-normalization of probability distributions based on the notion of case-based entropy Cc as a function of the cumulative probability c. Given a probability density p(x), Cc measures the diversity of the distribution up to a cumulative probability of c, by computing the length or support of an equivalent uniform distribution that has the same Shannon information as the conditional distribution of ^pc(x) up to cumulative probability c. We illustrate the utility of our approach by re-normalizing and comparing three well-known energy distributions in physics, namely, the Maxwell-Boltzmann, Bose-Einstein and Fermi-Dirac distributions for energy of sub-atomic particles. The comparison shows that Cc is a vast improvement over H as it provides a scale-free comparison of these diversity distributions and also allows for a comparison between parts of these diversity distributions.
    Original languageEnglish
    Article number8715605
    Number of pages10
    JournalComplexity
    Volume2017
    DOIs
    Publication statusPublished - 24 May 2017

    Fingerprint

    Entropy
    Probability distributions
    Biodiversity
    Physics
    Trajectories
    Economics

    Cite this

    Rajaram, R., Castellani, B., & Wilson, A. N. (2017). Advancing Shannon entropy for measuring diversity in systems. Complexity, 2017, [8715605]. https://doi.org/10.1155/2017/8715605
    Rajaram, Rajeev ; Castellani, Brian ; Wilson, A. N. / Advancing Shannon entropy for measuring diversity in systems. In: Complexity. 2017 ; Vol. 2017.
    @article{234f0b1967f342a683d297ce59c9cb14,
    title = "Advancing Shannon entropy for measuring diversity in systems",
    abstract = "From economic inequality and species diversity to power laws and the analysis of multiple trends and trajectories, diversity within systems is a major issue for science. Part of the challenge is measuring it. Shannon entropy H has been used to re-think diversity within probability distributions, based on the notion of information. However, there are two major limitations to Shannon's approach. First, it cannot be used to compare diversity distributions that have different levels of scale. Second, it cannot be used to compare parts of diversity distributions to the whole. To address these limitations, we introduce a re-normalization of probability distributions based on the notion of case-based entropy Cc as a function of the cumulative probability c. Given a probability density p(x), Cc measures the diversity of the distribution up to a cumulative probability of c, by computing the length or support of an equivalent uniform distribution that has the same Shannon information as the conditional distribution of ^pc(x) up to cumulative probability c. We illustrate the utility of our approach by re-normalizing and comparing three well-known energy distributions in physics, namely, the Maxwell-Boltzmann, Bose-Einstein and Fermi-Dirac distributions for energy of sub-atomic particles. The comparison shows that Cc is a vast improvement over H as it provides a scale-free comparison of these diversity distributions and also allows for a comparison between parts of these diversity distributions.",
    author = "Rajeev Rajaram and Brian Castellani and Wilson, {A. N.}",
    year = "2017",
    month = "5",
    day = "24",
    doi = "10.1155/2017/8715605",
    language = "English",
    volume = "2017",
    journal = "Complexity",
    issn = "1076-2787",
    publisher = "John Wiley and Sons Inc.",

    }

    Rajaram, R, Castellani, B & Wilson, AN 2017, 'Advancing Shannon entropy for measuring diversity in systems', Complexity, vol. 2017, 8715605. https://doi.org/10.1155/2017/8715605

    Advancing Shannon entropy for measuring diversity in systems. / Rajaram, Rajeev; Castellani, Brian; Wilson, A. N.

    In: Complexity, Vol. 2017, 8715605, 24.05.2017.

    Research output: Contribution to journalArticle

    TY - JOUR

    T1 - Advancing Shannon entropy for measuring diversity in systems

    AU - Rajaram, Rajeev

    AU - Castellani, Brian

    AU - Wilson, A. N.

    PY - 2017/5/24

    Y1 - 2017/5/24

    N2 - From economic inequality and species diversity to power laws and the analysis of multiple trends and trajectories, diversity within systems is a major issue for science. Part of the challenge is measuring it. Shannon entropy H has been used to re-think diversity within probability distributions, based on the notion of information. However, there are two major limitations to Shannon's approach. First, it cannot be used to compare diversity distributions that have different levels of scale. Second, it cannot be used to compare parts of diversity distributions to the whole. To address these limitations, we introduce a re-normalization of probability distributions based on the notion of case-based entropy Cc as a function of the cumulative probability c. Given a probability density p(x), Cc measures the diversity of the distribution up to a cumulative probability of c, by computing the length or support of an equivalent uniform distribution that has the same Shannon information as the conditional distribution of ^pc(x) up to cumulative probability c. We illustrate the utility of our approach by re-normalizing and comparing three well-known energy distributions in physics, namely, the Maxwell-Boltzmann, Bose-Einstein and Fermi-Dirac distributions for energy of sub-atomic particles. The comparison shows that Cc is a vast improvement over H as it provides a scale-free comparison of these diversity distributions and also allows for a comparison between parts of these diversity distributions.

    AB - From economic inequality and species diversity to power laws and the analysis of multiple trends and trajectories, diversity within systems is a major issue for science. Part of the challenge is measuring it. Shannon entropy H has been used to re-think diversity within probability distributions, based on the notion of information. However, there are two major limitations to Shannon's approach. First, it cannot be used to compare diversity distributions that have different levels of scale. Second, it cannot be used to compare parts of diversity distributions to the whole. To address these limitations, we introduce a re-normalization of probability distributions based on the notion of case-based entropy Cc as a function of the cumulative probability c. Given a probability density p(x), Cc measures the diversity of the distribution up to a cumulative probability of c, by computing the length or support of an equivalent uniform distribution that has the same Shannon information as the conditional distribution of ^pc(x) up to cumulative probability c. We illustrate the utility of our approach by re-normalizing and comparing three well-known energy distributions in physics, namely, the Maxwell-Boltzmann, Bose-Einstein and Fermi-Dirac distributions for energy of sub-atomic particles. The comparison shows that Cc is a vast improvement over H as it provides a scale-free comparison of these diversity distributions and also allows for a comparison between parts of these diversity distributions.

    U2 - 10.1155/2017/8715605

    DO - 10.1155/2017/8715605

    M3 - Article

    VL - 2017

    JO - Complexity

    JF - Complexity

    SN - 1076-2787

    M1 - 8715605

    ER -