在斯瓦希里语的核心:对自然语言处理数据收集方法和语料库策划的探索
1College of Information and Communication Technologies (CoICT), University of Dar Es Salaam, Ali Hassan Mwinyi Road, Kijitonyama campus, Dar Es Salaam TZ 33335, Tanzania.
Data in brief
|September 5, 2024
概括
为自然语言处理 (NLP) 任务创建了一个新的斯瓦希里语库,即斯瓦希里语句的数据集. 这种广泛的语料库将有助于开发和评估斯瓦希里语言的NLP算法.
科学领域:
- 计算语言学 计算语言学
- 语料库的语言学.
- 自然语言处理自然语言处理.
背景情况:
- 自然语言处理 (NLP) 需要特定语言的数据集来训练算法.
- 缺乏一个全面的斯瓦希里语库,用于为斯瓦希里语开发NLP工具.
研究的目的:
- 为NLP应用程序创建一个庞大的,多样化的斯瓦希里语库.
- 促进斯瓦希里语NLP模型的开发和评估.
主要方法:
- 系统地收集来自各种公共网站和报道的书面斯瓦希利语句子.
- 包括各种类别,如健康,商业,议会,宗教,教育,新闻,农业,社交媒体,非政府组织,政府,法律和政治.
- 数据的存档以及对来源真实性和语言丰富性的细致关注.
主要成果:
- 创建了一个斯瓦希里语语库数据集,包括1,693,228个句子,39,639,824个单词和871,452个唯一单词.
- 集体以文本文件格式出口,存储大小为168 MB.
- 确保了各种语言背景和与斯瓦希里语相关的主题的代表性.
结论:
- 斯瓦希里语库是研究人员,语言学家和开发人员的宝贵资源.
- 实现NLP任务的进步,如情感分析,文本分类和斯瓦希利语的机器翻译.
- 促进了斯瓦希里语特定NLP算法的开发,例如部分语音标记和命名实体识别.
更多相关视频
相关概念视频
Data Collection by Survey
6.4K
The systematic method of obtaining and analyzing accurate information of a population is called data collection. A survey is a standard method of data collection that involves collecting information from a target human population about their experience, opinion, or knowledge of a product, service, or process. The responses are recorded and interpreted. The most common survey examples are written questionnaires, face-to-face or telephonic conversations, focus groups, and electronic (e-mail or...
6.4K
Data Collection by Observations
11.9K
Data collection refers to a systematic way of obtaining, observing, measuring, and analyzing accurate information. Observational studies are one of the most widely used methods of data collection. It involves collecting data by observing the behavior and physical characteristics of a sample without making any modifications to the sample.
An astronomer viewing the motion and brightness of stars in the sky and recording the data is an example of observational data collection. A botanist recording...
An astronomer viewing the motion and brightness of stars in the sky and recording the data is an example of observational data collection. A botanist recording...
11.9K
Data Collection I
6.1K
Data collection gathers information needed to make accurate judgments about a patient's present condition. During a health history interview, subjective data is collected from the patient, their caregivers, or family members, and objective data is collected through observations and physical assessment. Patients are the primary source of subjective data. Thus information gathered from patients through interviews, observations, and physical examination is primary data. Secondary sources of...
6.1K
Data Collection II
8.1K
The nursing history captures and records the patient's health status, so that a care plan evolves to meet the patient's individual needs. The nursing health history is a part of the initial assessment. A comprehensive history covers all health dimensions and plays a significant role in the assessment process. A comprehensive history includes the patient's biographical information, reasons for seeking health care, expectations, present and past health history, medications, and...
8.1K
Data Collection III
2.7K
The physical assessment examines the patient for objective data that defines the patient's condition, and aids in formulating the nursing care plan. The purpose of physical assessment is a health status appraisal, which includes identifying health problems, and establishing a database for nursing intervention.
The principles to begin the physical assessment include conducting a comprehensive or problem-related history in a quiet, well-lit room, emphasizing privacy and comfort for the...
The principles to begin the physical assessment include conducting a comprehensive or problem-related history in a quiet, well-lit room, emphasizing privacy and comfort for the...
2.7K
Systematic Sampling Method
10.2K
Sampling is a technique to select a portion (or subset) of the larger population and study that portion (the sample) to gain information about the population. Data are the result of sampling from a population. The sampling method ensures that samples are drawn without bias and accurately represent the population. Because measuring the entire population in a study is not practical, researchers use samples to represent the population of interest.
Systematic sampling is one of the simplest methods...
Systematic sampling is one of the simplest methods...
10.2K


