Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving01:29

Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving

438
Mechanistic models play a crucial role in algorithms for numerical problem-solving, particularly in nonlinear mixed effects modeling (NMEM). These models aim to minimize specific objective functions by evaluating various parameter estimates, leading to the development of systematic algorithms. In some cases, linearization techniques approximate the model using linear equations.
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
438

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

A Single-Cell Atlas of Transcriptional and Immunoglobulin Repertoire Evolution in Early B Cell Development.

bioRxiv : the preprint server for biology·2025
Same author

GhostFold: Accurate protein structure prediction using structure-constrained synthetic coevolutionary signals.

bioRxiv : the preprint server for biology·2025
Same author

Data-optimal scaling of paired antibody language models.

bioRxiv : the preprint server for biology·2025
Same author

A curriculum learning approach to training antibody language models.

PLoS computational biology·2025
Same author

Longitudinal analysis of drift in the circulating human antibody repertoire over four years.

bioRxiv : the preprint server for biology·2025
Same author

A curriculum learning approach to training antibody language models.

bioRxiv : the preprint server for biology·2025

Related Experiment Video

Updated: May 5, 2026

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
03:14

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

1.3K

Evaluating Expert Specialization in Mixture-of-Experts Antibody Language Models.

Sarah M Burbach1, Simone Spandau1, Jonathan Hurtado1

  • 1Department of Immunology and Microbiology, The Scripps Research Institute, USA.

Biorxiv : the Preprint Server for Biology
|May 4, 2026
PubMed
Summary

Antibody language models (AbLMs) using a sparse Mixture-of-Experts (MoE) architecture improve learning of diverse antibody regions. This approach outperforms dense models, enhancing antibody sequence modeling capabilities.

More Related Videos

Developing a Salivary Antibody Multiplex Immunoassay to Measure Human Exposure to Environmental Pathogens
09:08

Developing a Salivary Antibody Multiplex Immunoassay to Measure Human Exposure to Environmental Pathogens

Published on: September 12, 2016

7.7K

Related Experiment Videos

Last Updated: May 5, 2026

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
03:14

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

1.3K
Developing a Salivary Antibody Multiplex Immunoassay to Measure Human Exposure to Environmental Pathogens
09:08

Developing a Salivary Antibody Multiplex Immunoassay to Measure Human Exposure to Environmental Pathogens

Published on: September 12, 2016

7.7K

Area of Science:

  • Computational biology
  • Bioinformatics
  • Artificial intelligence in medicine

Background:

  • Antibody language models (AbLMs) excel at learning antibody features but struggle with highly diverse, non-templated regions.
  • Current AbLMs utilize dense architectures where all parameters attend to every amino acid token.
  • Mixture-of-Experts (MoE) architectures, common in natural language processing, offer potential for specialized parameter use but are less explored in biological modeling.

Purpose of the Study:

  • To investigate the efficacy of a sparse Mixture-of-Experts (MoE) architecture for Antibody Language Models (AbLMs).
  • To adapt and optimize MoE routing strategies for antibody sequence data, particularly focusing on CDRH3 regions.
  • To evaluate if an MoE-based AbLM can outperform dense models in learning antibody features.

Main Methods:

  • Assessed existing MoE routing strategies, comparing token-choice and expert-choice routing for AbLMs.
  • Optimized the token-choice router to minimize padding token routing, enabling pre-training with variable sequence lengths.
  • Developed and trained a large-scale baseline antibody language model with a Top-2 MoE architecture (BALM-MoE) on diverse antibody sequences.

Main Results:

  • Token-choice routing strategies demonstrated superior performance over expert-choice routing in AbLMs, likely due to specialization in CDRH3 residues.
  • The optimized token-choice router effectively handled variable sequence lengths by minimizing padding token engagement.
  • The BALM-MoE model, with a Top-2 MoE architecture, outperformed its dense counterpart with an equivalent number of active parameters.

Conclusions:

  • Sparse MoE architectures are beneficial for AbLMs, enabling better learning of antibody sequence diversity compared to dense models.
  • Optimized MoE routing strategies enhance the applicability of AbLMs for biological sequence modeling.
  • MoE-based AbLMs represent a promising advancement for antibody design and analysis.