Related Experiment Video
Updated: May 31, 2025
03:14
Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
Published on: December 6, 2024
484
Evaluating human ability to distinguish between ChatGPT-generated and original scientific abstracts
Kylie J Nabata1,2, Yasir AlShehri3, Abdullah Mashat1,2
1Department of Surgery, St. Paul's Hospital, 1081 Burrard St., Vancouver, BC, V6Z 1Y6, Canada.
Updates in Surgery
|January 24, 2025
Summary
Human reviewers struggle to identify AI-generated scientific abstracts. A study found participants preferred ChatGPT abstracts, highlighting AI
Area of Science:
- Medical Education
- Artificial Intelligence in Research
- Scientific Communication
Background:
- The rapid advancement of artificial intelligence (AI) necessitates an understanding of its impact on academic writing.
- Evaluating the capabilities of AI tools like ChatGPT in generating scientific content is crucial for academic integrity.
Purpose of the Study:
- To assess the accuracy of human reviewers in distinguishing between AI-generated (ChatGPT) and original scientific abstracts.
- To explore reviewer preferences and perceptions regarding AI-generated content in academic writing.
Main Methods:
- An observational, cross-sectional study was conducted using an online survey.
- Participants (surgical trainees and faculty) were presented with one original and one ChatGPT-generated abstract.
- Respondents identified the AI-generated abstract and provided feedback on preferences and perceptions.
Main Results:
- Only 40% of participants correctly identified the original abstract.
- A significant majority (63.4%) preferred the ChatGPT-generated abstract (p=0.0001).
- Preference for the original abstract correlated with correct identification (OR 7.46).
Conclusions:
- Human reviewers demonstrated limited accuracy in differentiating AI-generated from original scientific abstracts.
- There is a discernible preference for AI-generated abstracts among academic reviewers.
- Findings underscore the need to address the implications and ethical considerations of AI in manuscript production.
Related Concept Videos
02:55Synthetic Biology
4.7K
Synthetic biology is an interdisciplinary science that involves using principles from disciplines such as engineering, molecular biology, cell biology, and systems biology. It involves remodeling existing organisms from nature or constructing completely new synthetic organisms for applications such as protein or enzyme production, bioremediation, value-added macromolecule production, and the addition of desirable traits to crops, to name a few.
Golden rice
Golden rice is a genetically modified...
Golden rice
Golden rice is a genetically modified...
4.7K
02:28Leaky Scanning
5.1K
During most eukaryotic translation processes, the small 40S ribosome subunit scans an mRNA from its 5' end until it encounters the first start AUG codon. The large 60S ribosomal subunit then joins the smaller one to initiate protein synthesis. The location of the translation initiation is largely determined by the nucleotides near the start codon as there may be multiple translation initiation sites present on the mRNA. Marilyn Kozak discovered that the sequence RCCAUGG (where R...
5.1K
02:18Types of Genetic Transfer Between Organisms
27.0K
Genetic transfer occurs when genetic information is passed from one organism to another. It occurs via two mechanisms: vertical gene transfer and horizontal gene transfer. Vertical gene transfer occurs when genetic information is transferred from one generation to the next, which happens much more frequently than horizontal gene transfer. Both sexual and asexual reproduction are forms of vertical gene transfer, where one or more organisms pass some or all of their genome onto their progeny.
27.0K
01:50Statistical Significance
20.1K
Once data is collected from both the experimental and the control groups, a statistical analysis is conducted to find out if there are meaningful differences between the two groups. A statistical analysis determines how likely any difference found is due to chance (and thus not meaningful). In psychology, group differences are considered meaningful, or significant, if the odds that these differences occurred by chance alone are 5 percent or less. Stated another way, if we repeated this...
20.1K
01:05Single Nucleotide Polymorphisms-SNPs
13.9K
A single nucleotide polymorphism or SNP is a single nucleotide variation at a specific genomic position in a large population. It is the most prevalent type of sequence variation found in the human genome. Point mutations that occur in more than 1% of the population qualify as SNPs. These are present once every 1000 nucleotides on an average in the human genome. Replacement of a purine with another purine (A/G) or a pyrimidine with another pyrimidine (C/T) is known as a transition. In contrast,...
13.9K
03:26Framing Effects
7.3K
Information is everywhere and its presentation—such as how and when items are presented—can impact our perceptions and decisions surrounding the info. This broad concept umbrellas framing effects—influences that occur due to the way information is framed in its appearance, whether it’s purely the order or the specific wording of a message. Let’s take a look at numerous ways in which two versions of something can objectively say the same thing, yet we respond in...
7.3K