在癌症注册表中自动检测 (几乎) 重复的内容文档
Tapio Niemi1, Jean Pierre Ghobril1, Gautier Defossez1
1Centre for Primary Care and Public Health (Unisanté), University of Lausanne, Lausanne, Switzerland.
International journal of medical informatics
|January 22, 2025
概括
这项研究开发了一种有效的方法,使用Simhash和Smith-Waterman算法识别重复和近重复的医疗文件. 该系统在大型数据集中成功检测到这些文件,改善了数据管理和研究准确性.
科学领域:
- 医疗信息学 医疗信息学
- 数据管理数据管理
- 计算生物学 计算生物学
背景情况:
- 重复和接近重复的医疗文件在数据管理,临床使用和研究方面带来了挑战.
- 癌症登记册中大量的多来源文件使处理复杂化.
- 对照对比对于在广泛的数据集中识别接近重复的数据是不可行的.
研究的目的:
- 开发和验证一个有效的系统,用于识别多源医疗文件中的完全重复和近重复.
- 解决基于人口的癌症登记处文件处理的挑战.
- 为了减少近重复检测中的错误阳性率,而不会影响灵敏度.
主要方法:
- 实现了一个结合Simhash (局部敏感哈希) 和史密斯-沃特曼文本对齐的系统.
- 在3042个手动验证的文档对上验证了该方法.
- 获得的高性能指标:AUC 0.96,灵敏度 0.92,特异性 1.00,PPV 1.00,NPV 0.91.
主要成果:
- 应用于224,398份医疗文件,识别了5.5%的重复和0.17-0.24%的几乎重复.
- 大多数近重复共享患者和发射器信息.
- 差异主要是行政方面的 (83%),而不是医疗方面的 (2%).
结论:
- 开发的方法可以有效地识别大型医疗文档集中的全部重复和大多数近重复.
- 该系统具有很高的准确性,可以在各种领域应用.
- 讨论了该方法的潜在改进和更广泛的应用.
更多相关视频
相关概念视频
02:50Combination Therapies and Personalized Medicine
4.8K
Combining two or more treatment methods increases the life span of cancer patients while reducing damage to vital organs or tissue from the overuse of a single treatment. Combination therapy also targets different cancer-inducing pathways, thus reducing the chances of developing resistance to treatment.
The combination of the drug acetazolamide and sulforaphane is a good example of combination therapy to treat cancer. The cells in the interior of a large tumor often die due to the hypoxic and...
The combination of the drug acetazolamide and sulforaphane is a good example of combination therapy to treat cancer. The cells in the interior of a large tumor often die due to the hypoxic and...
4.8K
02:26Comparing Copy Number Variations and SNPs
17.1K
Sequencing of the human genome has opened up several best-kept secrets of the genome. Scientists have identified thousands of genome variations that exist within a population. These variations can be a single nucleotide or a larger chromosomal variation.
Copy number variations or CNVs are the structural variations that cover more than 1kb of DNA sequence. The single nucleotide polymorphism (SNP), on the other hand, is a single nucleotide change or a point mutation that is found in more than 1%...
Copy number variations or CNVs are the structural variations that cover more than 1kb of DNA sequence. The single nucleotide polymorphism (SNP), on the other hand, is a single nucleotide change or a point mutation that is found in more than 1%...
17.1K