IRIS: Interpretable Retrieval-Augmented Classification for Long Interspersed Document Sequences.

Area of Science:

• Natural Language Processing
• Machine Learning
• Computational Linguistics

Background:

• Transformer models excel in document classification but face computational challenges with long texts.
• Existing methods for long-text processing offer partial solutions but are insufficient for exceptionally lengthy sequences.

Purpose of the Study:

• To propose IRIS (Interpretable Retrieval-Augmented Classification for long Interspersed Document Sequences), a novel, lightweight framework for efficient and interpretable long document classification.
• To enable processing of arbitrarily long documents without increased computational cost, suitable for single-GPU training.

Main Methods:

• IRIS segments documents into chunks and stores embeddings in a vector database.
• Learnable query vectors retrieve relevant chunks, and a linear attention mechanism aggregates embeddings for classification.
• The framework is designed for efficiency and interpretability, particularly for sparse, long-sequence data.

Main Results:

• IRIS achieves performance comparable to baseline models on standard benchmarks.
• The framework demonstrates superior performance in clinical note disease risk prediction tasks with extremely long and sparse documents.
• IRIS provides global interpretability by summarizing key risk factors identified by the model.

Conclusions:

• IRIS offers an efficient and interpretable solution for long document classification.
• The framework shows significant potential in healthcare applications requiring both high performance and explainability.
• IRIS addresses the limitations of current models in processing lengthy and complex textual data.