Deterministic binary vectors for efficient automated indexing of MEDLINE/PubMed abstracts

Area of Science:

• Computational biology
• Information science
• Biomedical informatics

Background:

• Automated indexing assists human indexers in managing the expanding biomedical literature.
• Random Indexing (RI) is an efficient method for generating document vector representations but faces scalability challenges in real-world systems.
• Current RI implementations require significant RAM for document vectors and a store of term vectors, hindering practical application.

Purpose of the Study:

• To develop and evaluate a deterministic binary variant of Random Indexing (RI).
• To address the scalability limitations of traditional RI methods for automated indexing.
• To enhance the efficiency and applicability of RI in biomedical literature indexing.

Main Methods:

• Development of a deterministic binary version of Random Indexing.
• Evaluation of the binary RI variant for document vector generation.
• Assessment of memory requirements and implementation feasibility compared to standard RI.

Main Results:

• The binary RI variant demonstrates increased capacity due to binary vectors.
• Elimination of the need to retain term vectors simplifies implementation.
• The method facilitates distributed implementations, significantly enhancing scalability.

Conclusions:

• The developed deterministic binary RI variant offers a scalable solution for automated biomedical literature indexing.
• This approach overcomes key implementation barriers of traditional RI, improving efficiency.
• The findings have positive implications for information retrieval and large-scale document analysis.