The minimizer Jaccard estimator is biased and inconsistent.

Area of Science:

• Bioinformatics
• Computational Biology
• Genomics

Background:

• Sequence sketching is crucial for efficient data processing in bioinformatics.
• Minimizer sketches offer scalability but raise concerns about accuracy and bias.

Purpose of the Study:

• To investigate the minimizer sketch for Jaccard similarity estimation.
• To analyze the bias and consistency of the minimizer Jaccard estimator.

Main Methods:

• Theoretical derivation of an analytical formula for minimizer sketch bias.
• Empirical validation of bias across various sequence families.
• Assessment of the impact of bias on the mashmap read mapping tool.

Main Results:

• The minimizer Jaccard estimator is proven to be biased and inconsistent.
• Bias is dependent on the layout of shared k-mers within sequences.
• Substantial bias observed, potentially doubling the estimate compared to the true Jaccard similarity.
• Demonstrated negative impact on mashmap read mapping accuracy.

Conclusions:

• The minimizer sketch introduces significant bias in Jaccard similarity estimation.
• This bias necessitates careful consideration when using minimizer sketches for sequence analysis.
• Findings have implications for the reliability of k-mer based bioinformatics tools.