Neural networks memorise personal information from one sample

Area of Science:

• Artificial Intelligence
• Medical Informatics
• Data Privacy

Background:

• Deep neural networks (DNNs) show high accuracy in large-scale disease diagnosis.
• Concerns exist regarding data privacy and algorithmic bias in DNN models.
• Patient data privacy is a critical consideration in medical AI development.

Purpose of the Study:

• To demonstrate that DNNs can memorize unique patient features (UFs) from training data.
• To explain the memorization phenomenon, particularly for rare concepts.
• To propose methods for detecting UF memorization in DNNs.

Main Methods:

• Analyzing the memorization of unique features (UFs) within DNNs.
• Investigating the correlation between concept rarity and memorization likelihood.
• Developing and validating methods to identify UF memorization without access to training data.

Main Results:

• DNNs can memorize UFs, such as names or IDs, even from single training samples.
• Memorization is more probable when UFs represent rare concepts within the dataset.
• Proposed methods effectively identify UF memorization in DNNs, usable by external entities.

Conclusions:

• UF memorization in DNNs has implications for model robustness and patient privacy.
• The risk of patient data leakage through memorized UFs is significant.
• External detection methods are crucial for mitigating privacy risks in deployed DNNs.