Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Types of Surveys01:27

Types of Surveys

214
Surveys are essential for marking property boundaries near water bodies. Different types of surveys are defined, each with its own function. Land surveys mark the property boundaries, while route surveys determine the position of properties on nearby highways. Topographic surveys create maps by capturing the three-dimensional features of the land. Hydrographic surveys focus on the shapes of underwater areas and the movement of streams through the properties. Mine surveys determine the relative...
214
Systematic Sampling Method01:17

Systematic Sampling Method

12.1K
Sampling is a technique to select a portion (or subset) of the larger population and study that portion (the sample) to gain information about the population. Data are the result of sampling from a population. The sampling method ensures that samples are drawn without bias and accurately represent the population. Because measuring the entire population in a study is not practical, researchers use samples to represent the population of interest.
Systematic sampling is one of the simplest methods...
12.1K
Self-Evaluation: Self-Enhancement and Self-Verification03:00

Self-Evaluation: Self-Enhancement and Self-Verification

5.6K
Social psychologists have documented that feeling good about ourselves and maintaining positive self-esteem is a powerful motivator of human behavior (Tavris & Aronson, 2008). In the United States, members of the predominant culture typically think very highly of themselves and view themselves as good people who are above average on many desirable traits (Ehrlinger, Gilovich, & Ross, 2005). Often, our behavior, attitudes, and beliefs are affected when we experience a threat to our...
5.6K
Decision Making: Traditional Method01:14

Decision Making: Traditional Method

4.8K
The process of hypothesis testing based on the traditional method includes calculating the critical value, testing the value of the test statistic using the sample data, and interpreting these values.
First, a specific claim about the population parameter is decided based on the research question and is stated in a simple form. Further, an opposing statement to this claim is also stated. These statements can act as null and alternative hypotheses, out of which a null hypothesis would be a...
4.8K
Feedback control systems01:26

Feedback control systems

556
Feedback control systems are categorized in various ways based on their design, analysis, and signal types.
Linear feedback systems are theoretical models that simplify analysis and design. These systems operate under the principle that their output is directly proportional to their input within certain ranges. For instance, an amplifier in a control system behaves linearly as long as the input signal remains within a specific range. However, most physical systems exhibit inherent nonlinearity...
556
Data Collection by Survey01:07

Data Collection by Survey

8.3K
The systematic method of obtaining and analyzing accurate information of a population is called data collection. A survey is a standard method of data collection that involves collecting information from a target human population about their experience, opinion, or knowledge of a product, service, or process. The responses are recorded and interpreted. The most common survey examples are written questionnaires, face-to-face or telephonic conversations, focus groups, and electronic (e-mail or...
8.3K

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

<sup>1</sup>H-NMR Analysis of Wine Metabolites: Method Development and Validation.

Molecules (Basel, Switzerland)·2026
Same author

Grounding spatial relations in text-only language models.

Neural networks : the official journal of the International Neural Network Society·2023
Same author

Increased top-down semantic processing in natural speech linked to better reading in dyslexia.

NeuroImage·2023
Same author

A Study on the Impacts of Slot Types and Training Data on Joint Natural Language Understanding in a Spanish Medication Management Assistant Scenario.

Sensors (Basel, Switzerland)·2022
Same author

Information retrieval and question answering: A case study on COVID-19 scientific literature.

Knowledge-based systems·2022
Same author

Lessons Learned from the Usability Evaluation of a Simulated Patient Dialogue System.

Journal of medical systems·2021
Same journal

Super greedy trees.

Artificial intelligence review·2026
Same journal

Artificial neural networks fighting real neural decline: a systematic review of AI in Alzheimer's research.

Artificial intelligence review·2026
Same journal

Topological data analysis and topological deep learning beyond persistent homology: a review.

Artificial intelligence review·2026
Same journal

Advances in artificial intelligence: a review for the creative industries.

Artificial intelligence review·2026
Same journal

Exploring unanswerability in machine reading comprehension: approaches, benchmarks, and open challenges.

Artificial intelligence review·2025
Same journal

Knowledge distillation and dataset distillation of large language models: emerging trends, challenges, and future directions.

Artificial intelligence review·2025
See all related articles

Related Experiment Video

Updated: Nov 19, 2025

A Protocol for Comprehensive Assessment of Bulbar Dysfunction in Amyotrophic Lateral Sclerosis ALS
12:43

A Protocol for Comprehensive Assessment of Bulbar Dysfunction in Amyotrophic Lateral Sclerosis ALS

Published on: February 21, 2011

35.5K

Survey on evaluation methods for dialogue systems.

Jan Deriu1, Alvaro Rodrigo2, Arantxa Otegi3

  • 1Zurich University of Applied Sciences (ZHAW), Steinberggasse 13, 8400 Winterthur, Switzerland.

Artificial Intelligence Review
|January 28, 2021
PubMed
Summary
This summary is machine-generated.

Evaluating dialogue systems is crucial but costly. This survey explores efficient methods for assessing task-oriented, conversational, and question-answering systems, reducing human labor and costs.

Keywords:
ChatbotsConversational AIDialogue systemsDiscourse modelEvaluation metrics

More Related Videos

Foreign Accent and Forensic Speaker Identification in Voice Lineups: The Influence of Acoustic Features Based on Prosody
09:09

Foreign Accent and Forensic Speaker Identification in Voice Lineups: The Influence of Acoustic Features Based on Prosody

Published on: September 27, 2024

668
Qualitative and Quantitative Validation of Tools with Rating Scales Aimed at Assessing the Quality of University Service-Learning
10:39

Qualitative and Quantitative Validation of Tools with Rating Scales Aimed at Assessing the Quality of University Service-Learning

Published on: August 29, 2025

572

Related Experiment Videos

Last Updated: Nov 19, 2025

A Protocol for Comprehensive Assessment of Bulbar Dysfunction in Amyotrophic Lateral Sclerosis ALS
12:43

A Protocol for Comprehensive Assessment of Bulbar Dysfunction in Amyotrophic Lateral Sclerosis ALS

Published on: February 21, 2011

35.5K
Foreign Accent and Forensic Speaker Identification in Voice Lineups: The Influence of Acoustic Features Based on Prosody
09:09

Foreign Accent and Forensic Speaker Identification in Voice Lineups: The Influence of Acoustic Features Based on Prosody

Published on: September 27, 2024

668
Qualitative and Quantitative Validation of Tools with Rating Scales Aimed at Assessing the Quality of University Service-Learning
10:39

Qualitative and Quantitative Validation of Tools with Rating Scales Aimed at Assessing the Quality of University Service-Learning

Published on: August 29, 2025

572

Area of Science:

  • Natural Language Processing
  • Human-Computer Interaction
  • Artificial Intelligence

Background:

  • Dialogue system development requires robust evaluation.
  • Current human evaluation methods are time- and cost-intensive.
  • Need for efficient and scalable evaluation techniques.

Purpose of the Study:

  • To survey existing methods for dialogue system evaluation.
  • To categorize evaluation techniques based on dialogue system type.
  • To identify cost-effective alternatives to human evaluation.

Main Methods:

  • Literature review of dialogue system evaluation techniques.
  • Classification of methods by dialogue system class (task-oriented, conversational, question-answering).
  • Analysis of technologies underpinning each dialogue system class.

Main Results:

  • Overview of diverse evaluation concepts and methodologies.
  • Detailed examination of evaluation approaches tailored to specific dialogue system types.
  • Identification of methods aimed at reducing human labor in evaluation.

Conclusions:

  • Effective evaluation is key to advancing dialogue system technology.
  • A variety of automated and semi-automated methods exist.
  • Selecting appropriate evaluation methods depends on the dialogue system's specific class and goals.