Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

Maximum likelihood set for estimating a probability mass function.

Bruno M Jedynak1, Sanjeev Khudanpur

  • 1Département de Mathématiques, Université des Sciences et Technologies de Lille, France. bruno.jedynak@jhu.edu

Neural Computation
|May 20, 2005
PubMed
Summary
This summary is machine-generated.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Association of infections and autoimmune conditions with cognition: a study using self-reported conditions and identifying a novel plasma biomarker.

medRxiv : the preprint server for health sciences·2026
Same author

Learning nonparametric ordinary differential equations from noisy data.

Journal of computational physics·2024
Same author

Longitudinal changes in Alzheimer's-related plasma biomarkers and brain amyloid.

Alzheimer's & dementia : the journal of the Alzheimer's Association·2023
Same author

Longitudinal changes in Alzheimer's-related plasma biomarkers and brain amyloid.

medRxiv : the preprint server for health sciences·2023
Same author

Multi-method investigation of factors influencing amyloid onset and impairment in three cohorts.

Brain : a journal of neurology·2022
Same author

Characterizing the Details of Spatial Construction: Cognitive Constraints and Variability.

Cognitive science·2022

We introduce a novel method to estimate probability mass functions (pmfs) from small samples. This approach defines a maximum likelihood set (MLS) and selects the optimal pmf using prior knowledge, achieving state-of-the-art results in language modeling.

Area of Science:

  • Statistics
  • Probability Theory
  • Machine Learning

Background:

  • Estimating probability mass functions (pmfs) for discrete variables from limited data is challenging.
  • Existing methods may not fully leverage observed data or incorporate prior knowledge effectively.

Purpose of the Study:

  • To develop a new method for estimating the probability mass function (pmf) of a discrete, finite random variable from small samples.
  • To define and characterize the Maximum Likelihood Set (MLS) of pmfs.

Main Methods:

  • The Maximum Likelihood Set (MLS) is defined as pmfs assigning more probability to observed counts than any other possible counts for a given sample size.
  • The MLS is geometrically characterized as a diamond-shaped subset of the probability simplex, bounded by hyper-planes.

Related Experiment Videos

  • A pmf is selected from the MLS by minimizing Kullback-Leibler distance to a prior knowledge pmf, solved using convex optimization.

    • Main Results:

    • The MLS is shown to contain the empirical distribution and Bayesian estimators like the Laplace estimator.
    • The proposed method successfully applies to language modeling, incorporating Zipf's law as prior knowledge.
    • State-of-the-art results were achieved in language modeling tasks.

    Conclusions:

    • The new method provides a robust way to estimate pmfs from small datasets.
    • The approach is conceptually simpler and performs competitively with existing methods in language modeling.
    • The framework allows for the principled incorporation of prior knowledge into pmf estimation.