NanoCellAnnotator: Formalizing Expert Cell Type Annotation with Large Language Models.

Area of Science:

• Computational Biology
• Genomics
• Bioinformatics

Background:

• Cell-type annotation in spatial transcriptomics is hindered by sparse gene panels, spatial heterogeneity, and lack of reference atlases.
• Current large language models (LLMs) for annotation can produce unsupported predictions and rely on cloud-hosted models, limiting reproducibility and privacy.
• Unconstrained LLM inference poses challenges for biologically accurate and deployable spatial transcriptomics analysis.

Purpose of the Study:

• To introduce NanoCellAnnotator, a novel framework for biologically constrained and confidence-aware automated cell-type annotation in spatial transcriptomics.
• To address limitations of existing LLM-based approaches by enabling local execution and incorporating biological constraints.
• To improve the accuracy, reproducibility, and interpretability of cell-type annotation in complex spatial transcriptomic data.

Main Methods:

• Spatial clusters identified using hybrid spatially regularized non-negative matrix factorization (hSNMF).
• Cluster markers mapped to ontology-derived functional programs via Gene Ontology enrichment and GO-slim projection.
• Lightweight, locally executable LLM performs constrained label selection using curated databases (PanglaoDB, CellMarker); confidence estimated via marker support and lineage separation.

Main Results:

• NanoCellAnnotator accurately recovers canonical cell populations in intrahepatic cholangiocarcinoma and breast cancer datasets with high confidence.
• The framework successfully identifies heterogeneous or transitional spatial domains as ambiguous, providing nuanced annotations.
• Annotation confidence metrics enable explicit flagging of ambiguous or heterogeneous clusters, enhancing interpretability.

Conclusions:

• NanoCellAnnotator provides a robust, reproducible, and privacy-preserving solution for spatial transcriptomics cell-type annotation.
• The biologically constrained and confidence-aware approach improves the reliability of automated annotations.
• This framework facilitates deeper understanding of cellular architecture in complex tissues.