Leveraging deep contrastive learning for semantic interaction.

Area of Science:

• Information Visualization
• Human-Computer Interaction
• Machine Learning

Background:

• Semantic interaction aims to understand user mental models during data exploration in visual analytics.
• Computational models can capture user intent and predict actions for enhanced system interaction.
• Deep learning offers powerful capabilities for learning complex functions, making it suitable for encoding user mental models.

Purpose of the Study:

• To demonstrate how deep contrastive learning can significantly improve semantic interaction in visual analytics systems.
• To develop a method for users to explore data arrangements while training a model of their evolving mental model.
• To apply and evaluate this approach within a practical visual analytics tool.

Main Methods:

• Utilized deep contrastive learning to train a parametric algorithm on user interactions with data visualizations.
• Enabled users to explore alternative data arrangements, facilitating the capture of their evolving mental models.
• Integrated the deep contrastive learning model into Z-Explorer, a visual analytics extension for Zotero.

Main Results:

• Deep contrastive learning effectively enhances semantic interaction in visual analytics.
• The approach successfully captures users' mental data models without explicit hyperparameter tuning.
• User studies confirmed the efficacy of the flexible approach in understanding user intent and data models.

Conclusions:

• Deep contrastive learning provides a powerful mechanism for improving semantic interaction and user modeling in visual analytics.
• The developed method offers an accessible way to capture complex user mental models, even for users without machine learning expertise.
• The Z-Explorer application demonstrates the practical utility and effectiveness of this approach in real-world data management scenarios.