Running field experiments using Facebook split test.

Area of Science:

• Business Research Methodology
• Experimental Design
• Online Research Methods

Background:

• Traditional experimental methods in business research offer high internal validity but suffer from artificiality and low managerial relevance.
• Field experiments address these limitations but are typically time- and resource-intensive.
• Existing online platforms like Amazon Mechanical Turk present challenges in data quality and sample representativeness.

Purpose of the Study:

• To present Facebook A/B split testing (FBST) as a time- and cost-effective online field experiment setting.
• To demonstrate FBST's capability for random assignment within ecologically valid stimuli.
• To compare FBST with laboratory settings and Amazon Mechanical Turk on key experimental design parameters.

Main Methods:

• Utilizing the Facebook A/B split test functionality for online field experiments.
• Embedding manipulated variables within ecologically valid stimuli for realistic testing.
• Comparative analysis of FBST against laboratory and Amazon Mechanical Turk settings.
• Empirical demonstration of FBST setup, pre-testing, execution, and analysis.

Main Results:

• FBST allows for random assignment in ecologically valid settings, enhancing managerial relevance.
• The method offers a practical alternative to traditional, resource-heavy field experiments.
• FBST provides a viable approach for conducting online field experiments efficiently.

Conclusions:

• Facebook A/B split testing presents a novel, efficient, and valid method for business research experiments.
• This approach overcomes limitations of artificiality and resource constraints found in other experimental settings.
• Researchers can leverage FBST for high-impact, relevant business insights through online field experiments.