Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

Stemming via distribution-based word segregation for classification and retrieval.

Narayan L Bhamidipati1, Sankar K Pal

  • 1Machine Intelligence Unit, Indian Statistical Institute, Kolkata 700108, India. bln_r@isical.ac.in

IEEE Transactions on Systems, Man, and Cybernetics. Part B, Cybernetics : a Publication of the IEEE Systems, Man, and Cybernetics Society
|April 10, 2007
PubMed
Summary
This summary is machine-generated.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Mental health problems raise the odds of cognitive impairment in COVID-19 survivors.

Frontiers in psychiatry·2024
Same author

Prediction and assessment of the impact of COVID-19 lockdown on air quality over Kolkata: a deep transfer learning approach.

Environmental monitoring and assessment·2022
Same author

Z-number-based AQI in rough set theoretic framework for interpretation of air quality for different thresholds of PM<sub>2.5</sub> and PM<sub>10</sub>.

Environmental monitoring and assessment·2022
Same author

Identifying Drug Resistant miRNAs Using Entropy Based Ranking.

IEEE/ACM transactions on computational biology and bioinformatics·2019
Same author

Fuzzy mutual information based grouping and new fitness function for PSO in selection of miRNAs in cancer.

Computers in biology and medicine·2017
Same author

Fuzzy-Rough Entropy Measure and Histogram Based Patient Selection for miRNA Ranking in Cancer.

IEEE/ACM transactions on computational biology and bioinformatics·2016
Same journal

Strategic Ability Updating in Concurrent Games by Coalitional Commitment.

IEEE transactions on systems, man, and cybernetics. Part B, Cybernetics : a publication of the IEEE Systems, Man, and Cybernetics Society·2015
Same journal

Meta-Analysis of the First Facial Expression Recognition Challenge.

IEEE transactions on systems, man, and cybernetics. Part B, Cybernetics : a publication of the IEEE Systems, Man, and Cybernetics Society·2012
Same journal

Adjustable model-based fusion method for multispectral and panchromatic images.

IEEE transactions on systems, man, and cybernetics. Part B, Cybernetics : a publication of the IEEE Systems, Man, and Cybernetics Society·2012
Same journal

Face Feature Weighted Fusion Based on Fuzzy Membership Degree for Video Face Recognition.

IEEE transactions on systems, man, and cybernetics. Part B, Cybernetics : a publication of the IEEE Systems, Man, and Cybernetics Society·2012
Same journal

A New Adaptive Fast Cellular Automaton Neighborhood Detection and Rule Identification Algorithm.

IEEE transactions on systems, man, and cybernetics. Part B, Cybernetics : a publication of the IEEE Systems, Man, and Cybernetics Society·2012
Same journal

Human-arm-and-hand-dynamic model with variability analyses for a stylus-based haptic interface.

IEEE transactions on systems, man, and cybernetics. Part B, Cybernetics : a publication of the IEEE Systems, Man, and Cybernetics Society·2012
See all related articles

This study introduces a new corpus-based method to refine word stemming, significantly improving text classification and information retrieval accuracy while reducing dictionary size. The technique groups similar words, enhancing data processing efficiency.

Area of Science:

  • Natural Language Processing
  • Information Retrieval
  • Machine Learning

Background:

  • Stemming algorithms are crucial for text normalization in NLP tasks.
  • Existing stemmers may lack optimal performance in classification and retrieval.
  • Dictionary size reduction is a key challenge in text processing.

Purpose of the Study:

  • To introduce a novel corpus-based method for stemmer refinement.
  • To enhance text classification and information retrieval performance.
  • To reduce dictionary size while maintaining or improving accuracy.

Main Methods:

  • Modeling words using multinomial distributions over corpus topics.
  • Employing sequential hypothesis testing to group distributionally similar words.

Related Experiment Videos

  • Developing a controllable refinement process with adjustable parameters.

    • Main Results:

    • Significant improvements in classification accuracy on 20 Newsgroups and WebKB datasets.
    • Demonstrated superiority over four existing methods in information retrieval tasks.
    • Achieved substantial dictionary size reduction without compromising performance.

    Conclusions:

    • The proposed stemmer refinement method offers significant advantages for text classification and retrieval.
    • The technique is adaptable and effective across different corpora (cross-corpus stemming).
    • The method provides a robust approach to optimizing stemming for enhanced NLP applications.