Federated learning-driven collaborative recommendation system for multi-modal art analysis and enhanced...

Area of Science:

• Artificial Intelligence
• Computer Science
• Digital Art Preservation

Background:

• Recommendation systems are prevalent but face challenges in the art domain, particularly concerning data privacy and copyright.
• Existing art similarity search methods struggle with these unique constraints.

Purpose of the Study:

• To propose a novel AI-based Collaborative Recommendation System (AICRS) framework for cross-institutional artwork similarity search and recommendation.
• To address data privacy and copyright concerns in art recommendation systems.

Main Methods:

• The AICRS framework utilizes multimodal data fusion, combining features extracted from image data using pre-trained Convolutional Neural Networks (CNN) and from text data using Bidirectional Encoder Representation from Transformers (BERT).
• A federated learning approach is employed, training models locally at each institution and aggregating parameters to optimize a global model, ensuring data privacy.

Main Results:

• The AICRS framework achieved a final accuracy of 92.02% on the SemArt dataset, significantly outperforming traditional CNN (81.52%) and Long Short-Term Memory (LSTM) models (83.44%).
• The AICRS framework demonstrated a lower final loss value of 0.1284 compared to CNN (0.248) and LSTM (0.188) models, indicating superior performance.

Conclusions:

• The AICRS framework offers an effective technical solution for artwork similarity search and recommendation.
• This research provides strong support for the practical recommendation and protection of artworks, especially in a cross-institutional context.