A comprehensive benchmarking for evaluating TCR embeddings in modeling TCR-epitope interactions.

Area of Science:

• Immunology
• Bioinformatics
• Computational Biology

Background:

• T cell receptor (TCR) sequences, especially complementarity-determining region 3 (CDR3), are complex.
• Machine learning applications in immunology require efficient TCR CDR3 embedding methods.
• Existing TCR CDR3 embedding strategies lack systematic evaluation, causing community confusion.

Purpose of the Study:

• To systematically evaluate and benchmark existing TCR CDR3 embedding methods.
• To compare the performance of different embedding strategies on various TCR downstream tasks.
• To provide a practical tool for selecting appropriate TCR CDR3 embedding models.

Main Methods:

• Extracted embedding models from 19 existing TCR CDR3 methods.
• Benchmarked models using four curated datasets and eight downstream classifiers.
• Assessed performance on TCR-epitope binding affinity prediction, epitope-specific TCR identification, clustering, and visualization.
• Utilized five downstream clustering methods and diverse performance metrics.

Main Results:

• Handcrafted TCR CDR3 embeddings demonstrated superior performance compared to data-driven methods for TCR-epitope interactions.
• Performance varied across different downstream tasks and datasets.
• Identified specific embedding models that excelled in particular immunological applications.

Conclusions:

• Systematic benchmarking clarifies the utility of various TCR CDR3 embedding strategies.
• Handcrafted embeddings are currently more effective for modeling TCR-epitope interactions.
• An all-in-one TCR CDR3 embedding package was developed to facilitate model selection for researchers.