Crowdsourcing the Measurement of Interstate Conflict

Area of Science:

• Political Science
• Computational Social Science
• Data Science

Background:

• Conflict data is often extracted from news reports, presenting a trade-off between the quality of expert coding and the speed/cost of machine coding.
• Existing methods for conflict data extraction from news face limitations in terms of cost, speed, and data quality.

Purpose of the Study:

• To introduce and evaluate a novel method for analyzing news documents to measure conflict.
• To address the limitations of expert and machine coding by integrating crowdsourcing with computational techniques.
• To improve the accuracy, cost-effectiveness, and speed of conflict data extraction from news sources.

Main Methods:

• Developed a hybrid approach combining crowdsourcing with computational methods for data extraction from news.
• Tested the new method on news documents related to Militarized Interstate Disputes.
• Compared the performance of the hybrid method against traditional expert and machine coding approaches.

Main Results:

• The crowdsourcing-supplemented method achieved an accuracy rate between 68 and 76 percent for analyzing conflict data.
• This accuracy represents a significant improvement over purely automated (machine) coding methods.
• The new approach demonstrated lower costs and faster processing times compared to expert coding.

Conclusions:

• A hybrid crowdsourcing and computational method offers a superior alternative for extracting conflict data from news reports.
• This approach effectively balances data accuracy with cost and speed considerations.
• The findings suggest a promising new direction for conflict data generation in social science research.