Leveraging Self-Supervised Vision Transformers for Segmentation-Based Transfer Function Design.

Area of Science:

• Computer Graphics
• Medical Imaging
• Artificial Intelligence

Background:

• Transfer functions are crucial for volume rendering, classifying structures and assigning optical properties.
• Designing transfer functions is often tedious and unintuitive, hindering efficient data exploration.
• Existing interactive and learning-based methods have limitations in speed and annotation requirements.

Purpose of the Study:

• To present a novel, interactive method for defining transfer functions in volume rendering.
• To leverage self-supervised pre-trained vision transformers for automated structure identification.
• To reduce the time and effort required for transfer function design and data annotation.

Main Methods:

• Utilized self-supervised pre-trained vision transformers for high-level feature extraction from volume data.
• Developed an interactive system where users select structures of interest in a slice viewer.
• Implemented an automated selection of similar structures based on extracted neural network features.

Main Results:

• The proposed method allows users to design transfer functions interactively within seconds.
• It significantly reduces the amount of necessary annotations by providing interactive classification feedback.
• The approach enables quick inference without requiring model retraining, unlike previous learning-based methods.

Conclusions:

• This novel method offers a fast and intuitive approach to transfer function design for volume rendering.
• Leveraging vision transformers streamlines the identification and classification of structures of interest.
• The system enhances interactive exploration of volume data by minimizing design and annotation time.