Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Expected Frequencies in Goodness-of-Fit Tests01:19

Expected Frequencies in Goodness-of-Fit Tests

A goodness-of-fit test is conducted to determine whether the observed frequency values are statistically similar to the frequencies expected for the dataset. Suppose the expected frequencies for a dataset are equal such as when predicting the frequency of any number appearing when casting a die. In that case, the expected frequency is the ratio of the total number of observations (n) to the number of categories (k).
Frequency-dependent Selection01:21

Frequency-dependent Selection

When the fitness of a trait is influenced by how common it is (i.e., its frequency) relative to different traits within a population, this is referred to as frequency-dependent selection. Frequency-dependent selection may occur between species or within a single species. This type of selection can either be positive—with more common phenotypes having higher fitness—or negative, with rarer phenotypes conferring increased fitness.Positive Frequency-Dependent SelectionIn positive...
Force Classification01:22

Force Classification

Forces play a crucial role in the study of physics and engineering. They are essential in describing the motion, behavior, and equilibrium of objects in the physical world. Forces can be classified based on their origin, type, and direction of action.
Contact and non-contact forces are two of the most widely used categories of forces. As the name suggests, contact forces require physical contact between two objects to act upon each other. Examples of contact forces include frictional,...
Classification of Systems-II01:31

Classification of Systems-II

Continuous-time systems have continuous input and output signals, with time measured continuously. These systems are generally defined by differential or algebraic equations. For instance, in an RC circuit, the relationship between input and output voltage is expressed through a differential equation derived from Ohm's law and the capacitor relation,
Aggregates Classification01:29

Aggregates Classification

Aggregate classification is generally based on its size, petrographic characteristics, weight, and source. Size classification ranges from coarse to fine aggregates, defined by the size of the particles. Coarse aggregates are particles that do not pass through ASTM sieve No. 4, and aggregates that pass through the sieve are fine aggregates.
Petrographic classification groups aggregates based on common mineralogical characteristics. Some of the common mineral groups found in aggregates are...
Relative Frequency Histogram01:14

Relative Frequency Histogram

The relative frequency depicts the proportion of data points that have each value. The frequency tells the number of data points that have each value. Like the histogram, a relative frequency histogram also has the same shape with a horizontal scale (the x-axis), but the vertical scale (the y-axis) is marked with relative frequencies (percentages of the whole) instead of actual frequencies. A relative frequency histogram is a graphical representation of a frequency distribution where the...

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Exploring Anti-Aging Literature via <i>ConvexTopics</i> and Large Language Models.

AMIA Joint Summits on Translational Science proceedings. AMIA Joint Summits on Translational Science·2026
Same author

Exploring Anti-Aging Literature via <i>ConvexTopics</i> and Large Language Models.

ArXiv·2026
Same author

AgentMD: Empowering language agents for risk prediction with large-scale clinical tool learning.

Nature communications·2025
Same author

AgentMD: Empowering Language Agents for Risk Prediction with Large-Scale Clinical Tool Learning.

ArXiv·2025
Same author

MedCPT: Contrastive Pre-trained Transformers with Large-scale PubMed Search Logs for Zero-shot Biomedical Information Retrieval.

ArXiv·2025
Same author

Ensuring Safety and Trust: Analyzing the Risks of Large Language Models in Medicine.

ArXiv·2025

Related Experiment Videos

The Ineffectiveness of Within - Document Term Frequency in Text Classification.

W John Wilbur1, Won Kim

  • 1National Center for Biotechnology Information, National Library of Medicine, National Institutes of Health, Bethesda, MD, U.S.A.

Information Retrieval
|October 6, 2009
PubMed
Summary

Text classification models often use word frequencies, but this study found simpler models ignoring word counts perform just as well. Word burstiness in documents limits the benefit of including local term frequency information.

Related Experiment Videos

Area of Science:

  • Natural Language Processing
  • Machine Learning
  • Information Retrieval

Background:

  • Document representation commonly uses a 'bag of words' model, including term frequencies.
  • Several classification models, such as Naïve Bayes (MM, MBM) and Support Vector Machines (SVM), utilize term frequencies.

Purpose of the Study:

  • To experimentally evaluate the necessity of incorporating local term frequency in text classification models.
  • To investigate if simplified models lacking term frequency information offer comparable performance.

Main Methods:

  • Comparative analysis of text classification models including Naïve Bayes (MM, MBM), Dirichlet Compound Multinomial (DCM), Exponential-Family Approximation (EDCM), and Support Vector Machines (SVM).
  • Experimental testing of model variants with and without local term frequency utilization.

Main Results:

  • Simplified models (MM, EDCM, SVM) that ignore word frequency performed comparably to their counterparts using frequency information.
  • A novel multivariate Bernoulli model (MBM) incorporating local term frequency showed no significant advantage over the basic MBM.

Conclusions:

  • Local term frequency does not significantly improve text classification performance in many common models.
  • The strong characteristic of word burstiness in documents may render additional occurrences of a word redundant for classification.