通过LLM生成以差异化和分布一致的过来增强差异化私有数据
Yiping Song1, Juhua Zhang2, Zhiliang Tian2
1College of Science, National University of Defense Technology, No.109, Deya Road, Kaifu District, Changsha, Hunan, 410073, China.
概括
本研究介绍了一种保护隐私的数据增强框架,使用大型语言模型和差异隐私. 它提高了私人域名的文本生成质量,同时保持了正式的隐私保证.
科学领域:
- 计算机科学 计算机科学
- 人工智能的人工智能
- 机器学习 机器学习
背景情况:
- 数据增强 (DA) 对于缓解数据不足至关重要,但在敏感领域存在隐私风险.
- 现有的保护隐私的文本生成方法缺乏正式保证.
- 不同隐私 (DP) 提供了理论上的保证,但往往会降低文本生成中的合成质量.
研究的目的:
- 开发一个新的基于DP的数据增强框架,用于私有领域的文本生成.
- 提高合成文本数据的实用性,同时确保正式的隐私保护.
- 解决现有DP方法在大规模文本生成中的局限性.
主要方法:
- 利用大型语言模型 (LLM) 来生成高质量的合成样本.
- 采用基于DP的区分器,通过知识蒸来构建,以选择适合领域的样本.
- 在低隐私预算下,利用基于DP的导师将标签分发与私有领域保持一致.
主要成果:
- 拟议的基于DP的DA框架有效生成高质量的,保护隐私的文本数据.
- 用DP合成的样本在实用性方面显著优于最先进的DP微调基线.
- 在三个医学文本分类数据集上的实证验证表明了卓越的性能.
结论:
- 基于DP的新型DA框架提供了一个强大的解决方案,用于在敏感域中保护隐私的文本生成.
- 这种方法成功地平衡了数据实用性和正式的隐私保证,优于现有的方法.
- 该方法在需要安全有效的数据增强的应用中显示出显著的前景.
相关概念视频
Data: Types and Distribution
1.8K
In biostatistics, data are the observations collected for analysis. There are two main types: parametric and non-parametric. Parametric data, which include continuous (e.g., weight) and discrete numerical data (e.g., number of tablets), assume a particular distribution pattern, often the normal distribution. Non-parametric data do not adhere to a specific distribution and typically comprise nominal (e.g., gender) and ordinal categorical data (e.g., pain scale ratings).
Distributions in...
Distributions in...
1.8K
Stereotypes, Prejudice, and Discrimination
95.4K
Humans are very diverse and although we share many similarities, we also have many differences. The social groups we belong to help form our identities (Tajfel, 1974). These differences may be difficult for some people to reconcile, which may lead to prejudice toward people who are different. Prejudice is a negative attitude and feeling toward an individual based solely on one’s membership in a particular social group (Allport, 1954; Brown, 2010). Prejudice is common against people who...
95.4K
Passive Filters
1.0K
Passive filters are utilized to shape the frequency spectrum of signals across a diverse array of applications. These filters, using only passive elements like resistors (R), inductors (L), and capacitors (C), are capable of selectively allowing or blocking certain frequency ranges without the need for external power sources.
Low-Pass Filters
Low-pass filters are designed to transmit signals with frequencies lower than the cutoff frequency, ωc, and attenuate those above it. The cutoff...
Low-Pass Filters
Low-pass filters are designed to transmit signals with frequencies lower than the cutoff frequency, ωc, and attenuate those above it. The cutoff...
1.0K
Model Approaches for Pharmacokinetic Data: Distributed Parameter Models
249
Pharmacokinetic models are mathematical constructs that represent and predict the time course of drug concentrations in the body, providing meaningful pharmacokinetic parameters. These models are categorized into compartment, physiological, and distributed parameter models.
The distributed parameter models are specifically designed to account for variations and differences in some drug classes. This model is particularly useful for assessing regional concentrations of anticancer or...
The distributed parameter models are specifically designed to account for variations and differences in some drug classes. This model is particularly useful for assessing regional concentrations of anticancer or...
249
Active Filters
1.3K
Active filters are electronic circuits that use operational amplifiers (op-amps), resistors, and capacitors to filter out unwanted frequency components from a signal. A first-order low-pass active filter is designed to pass signals with a frequency lower than a certain cutoff frequency and attenuate frequencies higher than that cutoff frequency. The transfer function for a first-order low-pass active filter is:
1.3K
Generalization, Discrimination, and Extinction
1.4K
Generalization, discrimination, and extinction are key concepts in operant conditioning that influence how behaviors are learned and maintained.
Generalization occurs when a behavior reinforced in one context is performed in similar situations. For instance, a student who studies diligently for calculus and receives excellent grades might apply the same study habits to psychology and history, expecting similar results. Generalization shows how learning in one setting can influence behavior in...
Generalization occurs when a behavior reinforced in one context is performed in similar situations. For instance, a student who studies diligently for calculus and receives excellent grades might apply the same study habits to psychology and history, expecting similar results. Generalization shows how learning in one setting can influence behavior in...
1.4K


