SEMbeddings:如何在使用大语言模型收集数据之前评估模型不适合
Tommaso Feraco1, Enrico Toffalini1
1Department of General Psychology, University of Padova, Padua, Italy.
Frontiers in psychology
|February 19, 2025
概括
大型语言模型 (LLM) 可以使用嵌入式来近似对象响应相关性. 一个名为SEMBeddings的新工具将这些模型与数据收集前评估的潜在测量模型集成在一起,帮助开发问卷.
科学领域:
- 心理测量 心理测量 心理测量
- 计算语言学 计算语言学
- 心理测量 心理测量
背景情况:
- 大型语言模型 (LLM) 显示了使用项嵌入和等号相似性近似实证相关矩阵的潜力.
- 评估模型匹配的传统方法通常在数据收集后出现,这可能导致问卷开发中的低效率.
研究的目的:
- 介绍SEMbeddings,这是一个新的工具,将微调的嵌入模型与潜在测量模型集成在一起.
- 在心理测量中收集数据之前评估模型的适合性或不适合性.
- 探索LLM衍生嵌入的实用性,以告知问卷开发.
主要方法:
- SEMbeddings将mpnet-个性模型与潜伏测量模型集成在一起.
- 应用SEMbeddings到VIA-IS-P (96个项目,24个字符强度) 使用来自31,697名参与者的答案.
- 在由mpnet-personality生成的等号相似度矩阵上进行了确认因素分析.
主要成果:
- 在嵌入小数点相似性和经验物品相关性之间发现了显著的相关性 (r=0.67).
- 传统的适合性指数可能会误导SEM嵌入,建议更保守的结论.
- 来自SEMBeddings的修改指数为潜在的项目不适应提供了有价值的见解.
结论:
- 在问卷开发中,SEMBeddings为数据收集前评估提供了一个有前途的方法.
- 从LLM衍生的程序可以提高新问卷开发的可靠性.
- 来自SEMBeddings的修改指数可以作为选项选择的选工具.
更多相关视频
相关概念视频
01:19Survival Tree
52
Survival trees are a non-parametric method used in survival analysis to model the relationship between a set of covariates and the time until an event of interest occurs, often referred to as the "time-to-event" or "survival time." This method is particularly useful when dealing with censored data, where the event has not occurred for some individuals by the end of the study period, or when the exact time of the event is unknown.
Building a Survival Tree
Constructing a...
Building a Survival Tree
Constructing a...
52
02:07Improving Translational Accuracy
2.5K
2.5K
01:02Quantifying and Rejecting Outliers: The Grubbs Test
1.4K
Sometimes, a data set can have a recorded numerical observation that greatly deviates from the rest of the data. Assuming that the data is normally distributed, a statistical method called the Grubbs test can be used to determine whether the observation is truly an outlier. To perform a two-tailed Grubbs test, first, calculate the absolute difference between the outlier and the mean. Then, calculate the ratio between this difference and the standard deviation of the sample. This...
1.4K
01:29Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving
38
Mechanistic models play a crucial role in algorithms for numerical problem-solving, particularly in nonlinear mixed effects modeling (NMEM). These models aim to minimize specific objective functions by evaluating various parameter estimates, leading to the development of systematic algorithms. In some cases, linearization techniques approximate the model using linear equations.
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
38
01:16Goodness-of-Fit Test
3.3K
The goodness-of-fit test is a type of hypothesis test which determines whether the data "fits" a particular distribution. For example, one may suspect that some anonymous data may fit a binomial distribution. A chi-square test (meaning the distribution for the hypothesis test is chi-square) can be used to determine if there is a fit. The null and alternative hypotheses may be written in sentences or stated as equations or inequalities. The test statistic for a goodness-of-fit test is given as...
3.3K
01:23Mechanistic Models: Compartment Models in Individual and Population Analysis
25
Mechanistic models are utilized in individual analysis using single-source data, but imperfections arise due to data collection errors, preventing perfect prediction of observed data. The mathematical equation involves known values (Xi), observed concentrations (Ci), measurement errors (εi), model parameters (ϕj), and the related function (ƒi) for i number of values. Different least-squares metrics quantify differences between predicted and observed values. The ordinary least...
25