BanglaDial:一个合并和不平衡的文本数据集用于孟加拉地区方言分析
Mehraj Hossain Mahi1, Anzir Rahman Khan1, Mayen Uddin Mojumdar1
1Multidisciplinary Action Research Lab, Department of Computer Science and Engineering, Daffodil International University, Birulia, Dhaka 1216, Bangladesh.
Data in brief
|November 11, 2025
概括
BanglaDial 是孟加拉语方言文本的新数据集,涵盖了11种方言. 这种资源支持自然语言处理 (NLP) 对于代表性不足的孟加拉语说者.
科学领域:
- 计算语言学 计算语言学
- 语料库语言学 语料库语言学
- 社会语言学 社会语言学
背景情况:
- 自然语言处理 (NLP) 技术主要使用标准孟加拉语,忽视了各种方言的变化.
- 孟加拉语由超过2.65亿人讲,在其区域方言中表现出显著的语音,词汇和语法差异.
- 现有的数字工具缺乏对孟加拉语代表性不足的方言种类的支持.
研究的目的:
- 介绍BanglaDial,一个统一的孟加拉语方言文本.
- 为解决孟加拉方言在NLP技术中的不足.
- 为了促进自动方言识别和开发方言意识的NLP应用程序.
主要方法:
- 从在线公共存储库中编制了一个由60,729个句子级别条目组成的数据集.
- 包括来自11个孟加拉地区方言 (例如,奇塔贡,西尔赫特,巴里萨尔,兰普尔) 和标准孟加拉语的样本.
- 用方言标签注释所有样本,捕捉语音,词汇和语法变化.
主要成果:
- 创建了BanglaDial,这是孟加拉语方言文本的综合体.
- 该语料库捕捉了11个地区方言和标准孟加拉语的语言分歧.
- 为NLP研究孟加拉语中代表性不足的品种提供了基础.
结论:
- BanglaDial弥合了孟加拉方言NLP资源上的差距.
- 该数据集支持孟加拉语使用者的语言保护和数字包容.
- 允许开发更公平,更全面的自然语言处理工具.
相关概念视频
01:12Regional Terms
14.2K
Regional terms describe anatomy by dividing the body parts into different regions that contain structures involved in contributing similar functions. Using these terms helps increase the accurate description and identification of the particular region of interest or region affected by the disease.
Primarily, the human body has two major regions, the axial and appendicular regions. The axial region comprises regions from the head to the abdomen and makes up the central body axis. In contrast,...
Primarily, the human body has two major regions, the axial and appendicular regions. The axial region comprises regions from the head to the abdomen and makes up the central body axis. In contrast,...
14.2K
01:23Test for Homogeneity
2.4K
The goodness–of–fit test can be used to decide whether a population fits a given distribution, but it will not suffice to decide whether two populations follow the same unknown distribution. A different test, called the test for homogeneity, can be used to conclude whether two populations have the same distribution. To calculate the test statistic for a test for homogeneity, follow the same procedure as with the test of independence. The hypotheses for the test for homogeneity can...
2.4K
01:17Region of Convergence
872
The z-transform is a powerful mathematical tool used in the analysis of discrete-time signals and systems. It is a crucial tool in the analysis of discrete-time systems, but its convergence is limited to specific values of the complex variable z. This range of values, known as the Region of Convergence (ROC), is fundamental in determining the behavior and stability of a system or signal. The ROC defines the region in the complex plane where the z-transform converges, which can take various...
872
01:11Regression Analysis
7.9K
Regression analysis is a statistical tool that describes a mathematical relationship between a dependent variable and one or more independent variables.
In regression analysis, a regression equation is determined based on the line of best fit– a line that best fits the data points plotted in a graph. This line is also called the regression line. The algebraic equation for the regression line is called the regression equation. It is represented as:
In regression analysis, a regression equation is determined based on the line of best fit– a line that best fits the data points plotted in a graph. This line is also called the regression line. The algebraic equation for the regression line is called the regression equation. It is represented as:
7.9K
01:28Lateralization
949
Brain lateralization refers to the division of mental processes and functions between the two hemispheres of the brain, a phenomenon that optimizes neural efficiency and underpins complex abilities in humans. This specialization allows each hemisphere to perform tasks where it has a comparative advantage, facilitating more refined cognitive capabilities across different domains.
949
02:07Improving Translational Accuracy
3.5K
3.5K