A scaling law to model the effectiveness of identification techniques

Area of Science:

• Artificial Intelligence
• Statistical Modeling
• Privacy Engineering

Background:

• Artificial intelligence (AI) is widely used for individual identification, but assessing its large-scale effectiveness and associated privacy risks is challenging.
• Existing methods for quantifying identification accuracy lack scalability and predictive power for real-world scenarios.

Purpose of the Study:

• To develop a principled framework for forecasting the privacy risks posed by AI-driven identification techniques.
• To create a scalable model for predicting the correctness of identification methods across different contexts.

Main Methods:

• Proposed a two-parameter Bayesian model for exact matching identification techniques.
• Derived an analytical expression for correctness (κ), representing the fraction of accurately identified individuals.
• Generalized the model to forecast the scalability of correctness for exact, sparse, and machine learning-based identification methods.

Main Results:

• The proposed two-parameter Bayesian model accurately fits 476 empirical correctness curves.
• The method demonstrates superior performance compared to traditional curve-fitting techniques and entropy-based rules of thumb.
• The model effectively forecasts how identification correctness scales from small-scale experiments to large-scale, real-world applications.

Conclusions:

• The developed Bayesian framework provides a robust method for assessing and forecasting privacy risks associated with AI identification systems.
• This work supports independent accountability for AI-based biometric systems by offering a quantifiable measure of identification effectiveness.
• The model's ability to generalize across various identification techniques highlights its utility for privacy risk assessment in diverse AI applications.