Measurement noise scaling laws for cellular representation learning.

Area of Science:

• Computational Biology
• Machine Learning
• Genomics

Background:

• Deep learning scaling laws typically relate performance to model and dataset size.
• Biological single-cell genomic data often suffers from measurement noise due to molecular undersampling.
• Understanding noise impact is crucial for reliable biological data analysis.

Purpose of the Study:

• To identify measurement noise as a new performance scaling axis in deep learning.
• To quantify the relationship between data quality and model performance in biological data.
• To explore the generalizability of noise-related scaling laws across different domains.

Main Methods:

• Introduced an information-theoretic metric for cellular representation model quality.
• Analyzed scaling relationships across various model types and datasets.
• Derived the scaling law from a Gaussian noise model.
• Validated the findings in image classification models with imaging noise.

Main Results:

• Identified a distinct logarithmic scaling law for measurement noise.
• Demonstrated that model quality scales with data sampling depth (data quality).
• Established a quantitative relationship consistent across different models and datasets.
• Showed similar noise scaling in image classification, suggesting a general phenomenon.

Conclusions:

• Measurement noise is a critical, quantifiable factor influencing deep learning performance.
• A universal scaling law with noise can guide data generation and curation strategies.
• This finding is particularly relevant for fields with high data variability, such as single-cell genomics and imaging.