CRD: Fast Co-clustering on Large Datasets Utilizing Sampling-Based Matrix Decomposition.

Area of Science:

• Data Mining and Machine Learning
• Bioinformatics
• Recommender Systems

Background:

• Co-clustering algorithms simultaneously group rows and columns of data matrices.
• Existing co-clustering methods face scalability issues due to high computational complexity (O(m × n)) and memory constraints.
• These limitations hinder the analysis of large-scale datasets common in text mining, microarray analysis, and recommendation systems.

Purpose of the Study:

• To propose a novel, efficient framework, CRD, for fast co-clustering of large datasets.
• To address the computational and memory limitations of traditional co-clustering algorithms.
• To enable effective co-clustering for datasets that cannot fit entirely into main memory.

Main Methods:

• CRD framework utilizes sampling-based matrix decomposition techniques.
• Achieves execution time linear with respect to the number of rows (m) and columns (n).
• Designed to handle datasets that exceed available main memory capacity.

Main Results:

• CRD demonstrates competitive accuracy compared to existing co-clustering algorithms.
• Achieves significantly reduced computational cost, making it suitable for large datasets.
• Successfully operates without requiring the entire data matrix to be loaded into memory.

Conclusions:

• CRD provides an efficient and scalable solution for co-clustering large datasets.
• Overcomes memory and computational bottlenecks of previous co-clustering approaches.
• Offers a practical method for uncovering hidden structures in massive data matrices across various domains.