Search research articles

ABOUT JoVE

Overview Leadership Blog JoVE Help Center

AUTHORS

Publishing Process Editorial Board Scope & Policies Peer Review FAQ Submit

LIBRARIANS

Testimonials Subscriptions Access Resources Library Advisory Board FAQ

RESEARCH

JoVE Journal Methods Collections JoVE Encyclopedia of Experiments Archive

EDUCATION

JoVE Core JoVE Business JoVE Science Education JoVE Lab Manual Faculty Resource Center Faculty Site

Terms & Conditions of Use

Related Concept Videos

Language Development

Language Development

Children master language quickly and with relative ease, supported by both biological predisposition and reinforcement. B. F. Skinner (1957) proposed that language is learned through reinforcement, while Noam Chomsky (1965) argued that language acquisition mechanisms are biologically determined.
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...

Improving Translational Accuracy

Improving Translational Accuracy

Language and Cognition

Language and Cognition

Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by

Same author

Scoping of Policy Impacts for Regulating E-cigarettes (SPIRE): findings from a data and decision analytic model mapping project.

NIHR open research·2026

Same author

Development and validation of a multi-agent AI pipeline for automated credibility assessment of tobacco misinformation: a proof-of-concept study.

Frontiers in artificial intelligence·2026

Same author

IQOS-marketing a tobacco product in the UK.

Tobacco control·2025

Same author

Developing and evaluating an educational intervention on conflicts of interest and corporate influence on science.

Health promotion international·2025

Same author

Impact of heated tobacco products on biomarkers of potential harm and adverse events: a systematic review and meta-analysis.

Tobacco control·2025

Same author

Effects of retail environment regulations on reducing tobacco retailers and operating hours: a case study in Egypt.

Tobacco control·2025

Search research articles

Related Experiment Video

Updated: Sep 17, 2025

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

Scalable evaluation framework for retrieval augmented generation in tobacco research using large Language models.

Sherif Elmitwalli¹, John Mehegan², Sophie Braznell²

¹Tobacco Control Research Group, Department for Health, University of Bath, Bath, UK. se606@bath.ac.uk.

Scientific Reports

|July 2, 2025

Summary

This summary is machine-generated.

A new framework evaluates large language models (LLMs) in retrieval-augmented generation (RAG) for tobacco research. Mixtral 8×7B outperformed Llama 3.1 70B, showing the need for domain-specific LLM assessment in public health.

Keywords:

AI evaluation Domain-Specific information retrieval Expert validation Goal-Question-Metric framework Large Language models Retrieval-Augmented generation

More Related Videos

Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems

Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems

Published on: June 13, 2025

Objectification of Tongue Diagnosis in Traditional Medicine, Data Analysis, and Study Application

Objectification of Tongue Diagnosis in Traditional Medicine, Data Analysis, and Study Application

Published on: April 14, 2023

Related Experiment Videos

Last Updated: Sep 17, 2025

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems

Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems

Published on: June 13, 2025

Objectification of Tongue Diagnosis in Traditional Medicine, Data Analysis, and Study Application

Objectification of Tongue Diagnosis in Traditional Medicine, Data Analysis, and Study Application

Published on: April 14, 2023

Area of Science:

Artificial Intelligence
Public Health Informatics
Computational Linguistics

Background:

Specialized knowledge domains require robust evaluation of retrieval-augmented generation (RAG) systems.
The tobacco research field lacks standardized frameworks for assessing large language models (LLMs), hindering accurate information retrieval for public health decisions.
Complex tobacco industry documentation necessitates domain-specific LLM evaluation.

Purpose of the Study:

To develop and validate a tobacco domain-specific evaluation framework for RAG systems.
To assess and compare the performance of different LLMs within RAG configurations using automated metrics and expert validation.
To establish a benchmark for LLM performance in specialized knowledge domains.

Main Methods:

Utilized a Goal-Question-Metric paradigm to evaluate LLMs (Mixtral 8×7B and Llama 3.1 70B) in RAG systems.
Incorporated automated assessments using GPT-4o and validation by three tobacco research specialists.
Employed a domain-specific dataset of 20 curated queries to measure performance across nine metrics, including accuracy, domain specificity, completeness, and clarity.

Main Results:

The developed framework successfully differentiated performance between LLM architectures.
Mixtral 8×7B significantly outperformed Llama 3.1 70B in accuracy (8.8/10 vs. 7.55/10) and domain specificity (8.65/10 vs. 7.6/10).
Hyperparameter optimization improved Mixtral's completeness, demonstrating the framework's utility for model refinement.

Conclusions:

Established a robust framework for evaluating LLMs in tobacco research RAG systems, applicable to other specialized domains.
Highlighted the critical importance of domain-specific evaluation for LLMs in public health applications due to significant performance variations.
Recommended extending the framework to broader corpora and additional LLMs for comprehensive assessment.