Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Convenience Sampling Method00:55

Convenience Sampling Method

11.8K
Sampling is a technique to select a portion (or subset) of the larger population and study that portion (the sample) to gain information about the population. Data are the result of sampling from a population. The sampling method ensures that samples are drawn without bias and accurately represent the population.
Convenience sampling is a non-random method of sample selection; this method selects individuals that are easily accessible and may result in biased data. For example, a marketing...
11.8K
Data Collection by Survey01:07

Data Collection by Survey

9.2K
The systematic method of obtaining and analyzing accurate information of a population is called data collection. A survey is a standard method of data collection that involves collecting information from a target human population about their experience, opinion, or knowledge of a product, service, or process. The responses are recorded and interpreted. The most common survey examples are written questionnaires, face-to-face or telephonic conversations, focus groups, and electronic (e-mail or...
9.2K
Data Collection by Observations01:08

Data Collection by Observations

15.2K
Data collection refers to a systematic way of obtaining, observing, measuring, and analyzing accurate information. Observational studies are one of the most widely used methods of data collection. It involves collecting data by observing the behavior and physical characteristics of a sample without making any modifications to the sample.
An astronomer viewing the motion and brightness of stars in the sky and recording the data is an example of observational data collection. A botanist recording...
15.2K
Surveys02:16

Surveys

17.1K
Often, psychologists develop surveys as a means of gathering data. Surveys are lists of questions to be answered by research participants, and can be delivered as paper-and-pencil questionnaires, administered electronically, or conducted verbally. Generally, the survey itself can be completed in a short time, and the ease of administering a survey makes it easy to collect data from a large number of people.
17.1K
Data Collection by Experiments01:13

Data Collection by Experiments

27.6K
Data collection is a systematic method of obtaining, observing, measuring, and analyzing accurate information. An experimental study is a standard method of data collection that involves the manipulation of the samples by applying some form of treatment prior to data collection. It refers to manipulating one variable to determine its changes on another variable. The sample subjected to treatment is known as “experimental units.”
An example of the experimental method is a public...
27.6K
Cluster Sampling Method01:20

Cluster Sampling Method

15.0K
Appropriate sampling methods ensure that samples are drawn without bias and accurately represent the population. Because measuring the entire population in a study is not practical, researchers use samples to represent the population of interest.
To choose a cluster sample, divide the population into clusters (groups) and then randomly select some of the clusters. All the members from these clusters are in the cluster sample. For example, if you randomly sample four departments from your...
15.0K

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Deconstructing Categorization in Visualization Recommendation: A Taxonomy and Comparative Study.

IEEE transactions on visualization and computer graphics·2021
Same author

Uncovering Effective Explanations for Interactive Genomic Data Analysis.

Patterns (New York, N.Y.)·2020
Same author

Three Lessons from Accelerating Scientific Insight Discovery via Visual Querying.

Patterns (New York, N.Y.)·2020
Same author

You can't always sketch what you want: Understanding Sensemaking in Visual Query Systems.

IEEE transactions on visualization and computer graphics·2019
Same author

Principles of Dataset Versioning: Exploring the Recreation/Storage Tradeoff.

Proceedings of the VLDB Endowment. International Conference on Very Large Data Bases·2017
Same author

Interactive Data Exploration with Smart Drill-Down.

Proceedings. International Conference on Data Engineering·2017
Same journal

Knowledge Graph and Large Language Model Co-learning via Structure-oriented Retrieval Augmented Generation.

Bulletin of the Technical Committee on Data Engineering·2026
See all related articles

Related Experiment Video

Updated: Feb 22, 2026

Methodology for Establishing a Community-Wide Life Laboratory for Capturing Unobtrusive and Continuous Remote Activity and Health Data
11:21

Methodology for Establishing a Community-Wide Life Laboratory for Capturing Unobtrusive and Continuous Remote Activity and Health Data

Published on: July 27, 2018

8.9K

Optimizing Open-Ended Crowdsourcing: The Next Frontier in Crowdsourced Data Management.

Aditya Parameswaran1, Akash Das Sarma2, Vipul Venkataraman1

  • 1University of Illinois.

Bulletin of the Technical Committee on Data Engineering
|September 28, 2017
PubMed
Summary
This summary is machine-generated.

This paper explores open-ended crowdsourcing for machine learning data. It details methods for optimizing this crucial, yet understudied, data generation technique.

More Related Videos

Selecting Multiple Biomarker Subsets with Similarly Effective Binary Classification Performances
07:35

Selecting Multiple Biomarker Subsets with Similarly Effective Binary Classification Performances

Published on: October 11, 2018

8.1K
Efficient Sampling of Genetically Encoded Biosensor Design Space Enabled with a Design of Experiments and Automation Workflow
08:58

Efficient Sampling of Genetically Encoded Biosensor Design Space Enabled with a Design of Experiments and Automation Workflow

Published on: October 17, 2025

707

Related Experiment Videos

Last Updated: Feb 22, 2026

Methodology for Establishing a Community-Wide Life Laboratory for Capturing Unobtrusive and Continuous Remote Activity and Health Data
11:21

Methodology for Establishing a Community-Wide Life Laboratory for Capturing Unobtrusive and Continuous Remote Activity and Health Data

Published on: July 27, 2018

8.9K
Selecting Multiple Biomarker Subsets with Similarly Effective Binary Classification Performances
07:35

Selecting Multiple Biomarker Subsets with Similarly Effective Binary Classification Performances

Published on: October 11, 2018

8.1K
Efficient Sampling of Genetically Encoded Biosensor Design Space Enabled with a Design of Experiments and Automation Workflow
08:58

Efficient Sampling of Genetically Encoded Biosensor Design Space Enabled with a Design of Experiments and Automation Workflow

Published on: October 17, 2025

707

Area of Science:

  • Computer Science
  • Artificial Intelligence
  • Data Management

Background:

  • Crowdsourcing is vital for generating large-scale training data for machine learning.
  • Sophisticated algorithms leverage crowdsourced data for automated applications.
  • Open-ended crowdsourcing is understudied but represents the next frontier in data management.

Purpose of the Study:

  • To survey and formally reason about optimizing open-ended crowdsourcing.
  • To address challenges in distilling consensus answers from disagreeing workers.
  • To explore methods for understanding diverse worker perspectives and selecting appropriate operators.

Main Methods:

  • Literature review of formal reasoning approaches in crowdsourcing.
  • Analysis of challenges in open-ended task aggregation.
  • Case studies and practical experience in optimizing open-ended crowdsourcing workflows.

Main Results:

  • Identification of key challenges in open-ended crowdsourcing, including answer aggregation and worker perspective analysis.
  • Description of effective approaches for managing and optimizing open-ended tasks.
  • Insights into selecting appropriate operators for diverse open-ended problems.

Conclusions:

  • Open-ended crowdsourcing requires specialized methods for data aggregation and quality control.
  • Formal reasoning and optimization are essential for unlocking the potential of this data management approach.
  • Further research is needed to fully address the complexities of open-ended crowdsourced data.