FLACON: An Information-Theoretic Approach to Flag-Aware Contextual Clustering for Large-Scale Document Organization

Area of Science:

• Information Science
• Computer Science
• Data Mining

Background:

• Traditional document clustering methods overlook crucial organizational context like priority and status.
• Existing context-aware systems often lack domain-specific intelligence.
• Large Language Model (LLM) based clustering demands substantial computational resources.

Purpose of the Study:

• Introduce Flag-Aware Context-sensitive Clustering (FLACON) to integrate multi-dimensional document context.
• Formalize document clustering as an entropy minimization problem.
• Address limitations of existing enterprise document organization solutions.

Main Methods:

• Developed FLACON, an information-theoretic approach using a six-dimensional flag system (Type, Domain, Priority, Status, Relationship, Temporal).
• Employed a composite distance function combining semantic content, contextual flags, and temporal factors.
• Utilized adaptive hierarchical clustering with efficient incremental updates for scalability.

Main Results:

• FLACON demonstrated a 7.8-fold improvement in clustering quality (Silhouette Score: 0.311 vs. 0.040) over traditional methods.
• Achieved 89% of GPT-4's quality while being approximately 7x faster for large datasets.
• Exhibited O(m log n) complexity for incremental updates and deterministic behavior suitable for compliance.

Conclusions:

• FLACON offers a robust and efficient solution for context-aware enterprise document clustering.
• The approach is practical for large-scale organization across diverse document types (emails, technical, financial).
• FLACON balances high clustering quality with computational efficiency and deterministic outcomes.