keymodels
Business experiments/experimental design/AB testing
When and how should business experiments/experimental design/ab testing be applied?
Contents
Business experiments, experimental design and AB testing are all techniques for testing the validity of something – be that a strategic hypothesis, new product packaging or a marketing approach.
Business experiments use controlled comparison to test a strategic assumption, product change, package design or marketing treatment before a full commitment is made. Experimental design defines how the evidence will be produced; A/B testing is one common form comparing two variants.
When to use it
Innovation and strategic change rely on assumptions that customers may not confirm. A well-designed experiment reduces uncertainty by exposing a limited population to alternatives and measuring the difference before the organisation scales one choice.
Use experimentation when two or more viable options exist, a wrong decision would be costly and the outcome can be measured safely on a smaller scale. Results can identify the stronger option and reveal how to refine it.
Experiments can answer:
- Which treatment causes a meaningful increase in sales?
- Which product should be released first?
- Which offer do target customers prefer in behaviour, not only in stated opinion?
- Which campaign produces the highest incremental response?
- Which recruitment channel generates the best qualified hires at acceptable cost?
Origins
Controlled experiments have roots in scientific and agricultural research. During the 1920s, statistician Ronald A. Fisher formalised randomisation, replication and experimental design at Rothamsted Experimental Station. Randomised controlled trials later became central to medicine and social science. Businesses adopted direct-mail and market tests throughout the twentieth century, while digital products made large-scale A/B tests faster and cheaper from the 1990s onward. The underlying logic remains counterfactual: estimate what changed because of the treatment by comparing otherwise equivalent groups.
What it is
“Business experiment” is the broad term. Experimental design specifies treatments, assignment, samples and measurement; A/B testing compares a control A with a changed version B and is common in digital and direct marketing. Whatever the label, the objective is maximum decision-relevant information with minimum bias. A television pilot applies a looser version of the same principle by testing audience response before financing a full series.
How to use it
Free account access
Read the full article.
Create your free KeyModels account to finish this article, save it to your library and keep your reading progress across devices.