Related Experiment Video
Updated: Jun 20, 2026
10:56
A User-friendly and Powerful R Analysis of Large-scale Datasets
Published on: November 4, 2025
AI and social science: Automatic classification tools for big data analysis in sociological research.
Andrea Nucita1, Assunta Penna1, Antonia Cava1
1Department of Cognitive Sciences, Psychology, Education, and Cultural Studies, University of Messina, Messina, Italy.
Plos One
|June 18, 2026
Summary
Automated classification tools show reliability comparable to human coders for analyzing social media data. Integrating AI enhances sociological analysis of public institutional communication on social network sites.
Area of Science:
- Sociology
- Computational Social Science
- Communication Studies
Background:
- Social Network Sites (SNS) are vital for public institutional communication.
- Automated classification tools, including Large Language Models (LLMs), are increasingly used for social media data analysis.
- A research gap exists in systematically assessing the reliability of AI-based categorizations against human coding, especially for semantically ambiguous categories.
Purpose of the Study:
- To compare the reliability of AI-generated classifications with human expert coding for public institutional communication on SNS.
- To determine if human-machine agreement in classifying social media data is comparable to inter-human coder agreement.
- To explore the potential and challenges of automated classification tools in sociological data analysis.
Main Methods:
- A case study was conducted on Facebook posts from two Italian universities (March 2020-March 2023).
- Posts were classified into eight categories of public institutional communication.
- Three researchers independently annotated the data, serving as a benchmark to compare with AI-based system classifications.
Main Results:
- Substantial interpretive ambiguity was identified across several communication categories, reflected in variability among human coders.
- Automated models achieved agreement with human classifications that was broadly comparable to inter-coder agreement.
- The study highlights the potential for AI to match human coding reliability, even with ambiguous data.
Conclusions:
- AI-based classification tools demonstrate a reliability comparable to human coders in analyzing social media data.
- Hybrid workflows integrating AI as an additional coder can enable scalable and transparent sociological analysis.
- The findings support the use of AI to overcome challenges in analyzing complex social media data for institutional communication.
Related Concept Videos
01:28Automatic Processing and Automatic Social Behavior
Automatic processing refers to the cognitive operations that occur without conscious intent or awareness, playing a fundamental role in shaping social cognition and behavior. These processes enable individuals to navigate complex social environments efficiently by relying on mental shortcuts and pre-existing knowledge structures known as schemas. One of the most influential mechanisms underlying automatic processing is priming, which subtly activates mental representations through exposure to...
01:22Statistical Package for the Social Sciences (SPSS)
The Statistical Package for the Social Sciences, or SPSS, is a data management and analysis software suite. Developed by SPSS Inc. in 1968 and acquired by IBM in 2009, this tool was initially designed for social science data analysis, evolving to serve a wider range of disciplines. It was later renamed to Statistical Product and Service Solutions.
SPSS streamlines the process from data preparation to analysis and reporting. It is characterized by its user-friendly interface, which conceals...
SPSS streamlines the process from data preparation to analysis and reporting. It is characterized by its user-friendly interface, which conceals...
01:29Econometric Views (EViews)
Econometric Views, often stylized as EViews, is a package that merges statistical analysis with econometric studies. It is designed to provide tools for time series analysis, forecasting, and econometric model simulation. The software originated from MicroTSP software and has evolved significantly since its inception in 1981. The history of EViews is marked by a continuous effort to enhance its computational speed and user interface. It was initially developed for large computing systems but...
01:11Statistical Analysis: Overview
When we take repeated measurements on the same or replicated samples, we will observe inconsistencies in the magnitude. These inconsistencies are called errors. To categorize and characterize these results and their errors, the researcher can use statistical analysis to determine the quality of the measurements and/or suitability of the methods.
One of the most commonly used statistical quantifiers is the mean, which is the ratio between the sum of the numerical values of all results and the...
One of the most commonly used statistical quantifiers is the mean, which is the ratio between the sum of the numerical values of all results and the...
01:26Classification of Systems-I
Linearity is a system property characterized by a direct input-output relationship, combining homogeneity and additivity.
Homogeneity dictates that if an input x(t) is multiplied by a constant c, the output y(t) is multiplied by the same constant. Mathematically, this is expressed as:
Homogeneity dictates that if an input x(t) is multiplied by a constant c, the output y(t) is multiplied by the same constant. Mathematically, this is expressed as:
01:29Aggregates Classification
Aggregate classification is generally based on its size, petrographic characteristics, weight, and source. Size classification ranges from coarse to fine aggregates, defined by the size of the particles. Coarse aggregates are particles that do not pass through ASTM sieve No. 4, and aggregates that pass through the sieve are fine aggregates.
Petrographic classification groups aggregates based on common mineralogical characteristics. Some of the common mineral groups found in aggregates are...
Petrographic classification groups aggregates based on common mineralogical characteristics. Some of the common mineral groups found in aggregates are...