DETEXA:通过SQL进行声明式可扩展文本探索和分析
Yannis Foufoulas1,2, Eleni Zacharia1,2, Harry Dimitropoulos1,2
1Department of Informatics and Telecommunications, National and Kapodistrian University of Athens, Panepistimiopolis, 15784 Ilisia, Greece.
International journal on digital libraries
|June 26, 2023
概括
本研究介绍了一种使用扩展的SQL用于数字图书馆的新型文本分析框架. 该框架加速元数据丰富和文本挖掘,比现有方法快三倍.
科学领域:
- 数字图书馆 数字图书馆
- 文本挖掘 (Text Mining) 是一个很好的方法.
- 数据库管理系统 数据库管理系统
背景情况:
- 对数字图书馆来说,元数据的丰富至关重要,因为数字图书馆面临着越来越多的开放访问出版物的挑战.
- 非结构化,大规模和异构的数据源对传统的文本挖掘构成重大障碍.
- 现有的方法往往缺乏端到端文本挖掘管道的效率和集成.
研究的目的:
- 引入一个可扩展的文本分析框架,在扩展的SQL中实现.
- 为了实现高性能,端到端的文本挖掘管道,包括数据收集,清理,处理和分析.
- 为了利用SQL的声明性质,为领域专家进行快速实验和API开发.
主要方法:
- 在扩展的SQL中开发了一个文本分析框架.
- 利用了现代数据库管理系统的可扩展性.
- 将数据收集,清理,处理和文本分析整合到一个统一的管道中.
主要成果:
- 拟议的框架显示了显著的加快速度,在常见的用例中实现高达三倍更快的性能.
- 实验分析证实了该框架在处理大型非结构化数据集方面的有效性.
- 基于SQL的方法使工作流更容易修改和API集成.
结论:
- 扩展的SQL框架为数字图书馆中的元数据丰富提供了有效和高性能解决方案.
- 这种方法解决了开放获取出版物的指数级增长所带来的挑战.
- 该系统使领域专家能够通过图形界面高效地管理文本挖掘工作流程.
相关概念视频
Extraction: Advanced Methods
494
Metal ions can be separated from one another by complexation with organic ligands–the chelating agent– to form uncharged chelates. Here, the chelating agent must contain hydrophobic groups and behave as a weak acid, losing a proton to bind with the metal. Since most organic ligands used in this process are insoluble or undergo oxidation in the aqueous phase, the chelating agent is initially added to the organic phase and extracted into the aqueous phase. The metal-ligand complex is...
494
Overview of Microsoft Excel as a Data Analysis Tool
594
Microsoft Excel is a cornerstone tool for data analysis and statistical operations, offering a wide array of functionalities to manage, analyze, and visualize data efficiently. Recognized for its versatility, Excel facilitates the performance of basic to complex statistical operations, serving as an indispensable asset for analysts, researchers, and students alike. Excel's significance in data analysis emanates from its spreadsheet environment, where data can be organized in rows and...
594
Econometric Views (EViews)
176
Econometric Views, often stylized as EViews, is a package that merges statistical analysis with econometric studies. It is designed to provide tools for time series analysis, forecasting, and econometric model simulation. The software originated from MicroTSP software and has evolved significantly since its inception in 1981. The history of EViews is marked by a continuous effort to enhance its computational speed and user interface. It was initially developed for large computing systems but...
176
Statistical Analysis: Overview
6.7K
When we take repeated measurements on the same or replicated samples, we will observe inconsistencies in the magnitude. These inconsistencies are called errors. To categorize and characterize these results and their errors, the researcher can use statistical analysis to determine the quality of the measurements and/or suitability of the methods.
One of the most commonly used statistical quantifiers is the mean, which is the ratio between the sum of the numerical values of all results and the...
One of the most commonly used statistical quantifiers is the mean, which is the ratio between the sum of the numerical values of all results and the...
6.7K
Statistical Analysis System (SAS)
238
SAS, short for Statistical Analysis System, is a powerful data analysis, management, and visualization tool. Developed by the SAS Institute in the early 1970s, SAS has evolved into a comprehensive software suite used across various industries for statistical analysis, business intelligence, and predictive modeling.
Applications: SAS finds applications in numerous fields, including healthcare for clinical trial analysis, finance for risk assessment, marketing for customer data analysis, and...
Applications: SAS finds applications in numerous fields, including healthcare for clinical trial analysis, finance for risk assessment, marketing for customer data analysis, and...
238
Statistical Package for the Social Sciences (SPSS)
407
The Statistical Package for the Social Sciences, or SPSS, is a data management and analysis software suite. Developed by SPSS Inc. in 1968 and acquired by IBM in 2009, this tool was initially designed for social science data analysis, evolving to serve a wider range of disciplines. It was later renamed to Statistical Product and Service Solutions.
SPSS streamlines the process from data preparation to analysis and reporting. It is characterized by its user-friendly interface, which conceals...
SPSS streamlines the process from data preparation to analysis and reporting. It is characterized by its user-friendly interface, which conceals...
407


