Downscaling BERT for Efficient Semantic Search of Clinical Terminology

High latency in large neural networks, such as bidirectional encoder representations from transformers (BERT), poses a challenge for real-time applications. This study explores reducing the size of BERT through model distillation using DistilBioBERT, CompactBioBERT, and TinyBioBERT and by removing layers, with the goal of lowering computational and memory requirements for efficient semantic search of clinical terminology. Using a newly developed clinical dataset, the performance of these compact models is evaluated against pre-trained biomedical BERT variants with progressively fewer layers.