field experiments disadvantages

10.2 Pros and Cons of Field Research

Learning objectives.

• Identify and explain the strengths of field research.
• Identify and explain the weaknesses of field research.

Field research has many benefits, as well as a set of drawbacks. We’ll explore both here.

Strengths of Field Research

Field research allows researchers to gain firsthand experience and knowledge about the people, events, and processes that they study. No other method offers quite the same kind of closeup lens on everyday life. This close-up on everyday life means that field researchers can obtain very detailed data about people and processes, perhaps more detailed than they can obtain using any other method.

Field research is an excellent method for understanding the role of social context in shaping people’s lives and experiences. It enables a greater understanding of the intricacies and complexities of daily life. Field research may also uncover elements of people’s experiences or of group interactions of which we were not previously aware. This in particular is a unique strength of field research. With other methods, such as interviews and surveys, we certainly can’t expect a respondent to answer a question to which they do not know the answer or to provide us with information of which they are not aware. And because field research typically occurs over an extended period of time, social facts that may not even be immediately revealed to a researcher but that become discovered over time can be uncovered during the course of a field research project.

In sum, the major benefits of field research are the following:

• It yields very detailed data.
• It emphasizes the role and relevance of social context.
• It can uncover social facts that may not be immediately obvious or of which research participants may be unaware.

Weaknesses of Field Research

Earlier I described the fact that field researchers are able to collect very detailed data as a benefit of this method. This benefit, however, does come at a cost. Because a field researcher’s focus is so detailed, it is by necessity also somewhat narrow. Field researchers simply are not able to gather data from as many individuals as, say, a survey researcher can reach. Indeed, field researchers generally sacrifice breadth in exchange for depth. Related to this point is the fact that field research is extremely time intensive.

Field research can also be emotionally taxing. In Chapter 9 "Interviews: Qualitative and Quantitative Approaches" , I assert that interview research requires, to a certain extent, the development of a relationship between a researcher and her participants. But if interviews and field research both require relationship development, you might say that interviews are more like casual dating while field research is more like a full-blown, committed marriage.

The relationships you develop as a field researcher are sustained over a much longer period than the hour or two it might take you to conduct an interview. Not only do the relationships last longer, but they are also more intimate. A number of field researchers have documented the complexities of relationships with research participants (Arditti, Joest, Lambert-Shute, & Walker, 2010; Keinman & Copp, 1993; MacLeod, 1995). MacLeod, J. (1995). On the making of ain’t no makin’ it. In J. MacLeod (Ed.), Ain’t no makin’ it: Aspirations and attainment in a low-income neighborhood (pp. 270–302). Boulder, CO: Westview Press; Arditti, J. A., Joest, K. A., Lambert-Shute, J., & Walker, L. (2010). The role of emotions in fieldwork: A self-study of family research in a corrections setting. The Qualitative Report, 15, 1387–1414; Keinman, S., & Copp, M. A. (1993). Emotions and fieldwork . Newbury Park, CA: Sage. On the plus side, these relationships can be very rewarding (and yield the rich, detailed data noted as a strength in the preceding discussion). But, as in any relationship, field researchers experience not just the highs but also the lows of daily life and interactions. And participating in day-to-day life with one’s research subjects can result in some tricky ethical quandaries (see Chapter 3 "Research Ethics" for a discussion of some of these quandaries). It can also be a challenge if your aim is to observe as “objectively” as possible.

Finally, documentation can be challenging for field researchers. Where survey researchers have the questionnaires participants complete and interviewers have recordings, field researchers generally have only themselves to rely on for documenting what they observe. This challenge becomes immediately apparent upon entering the field. It may not be possible to take field notes as you observe, nor will you necessarily know which details to document or which will become the most important details to have noted. And when you take notes after some observation, you may not recall everything exactly as you saw it when you were there.

In sum, the weaknesses of field research include the following:

• It may lack breadth; gathering very detailed information means being unable to gather data from a very large number of people or groups.
• It may be emotionally taxing.
• Documenting observations may be more challenging than with other methods.

Key Takeaways

• Strengths of field research include the fact that it yields very detailed data, it is designed to pay heed to social context, and it can uncover social facts that are not immediately obvious.
• Weaknesses of field research include that researchers may have to sacrifice breadth for depth, the possibility that the research will be emotionally taxing, and the fact that documenting observations can be challenging.
• In your opinion, what is the most important strength of field research? What do you view as its greatest weakness? Explain your position.
• Find an article reporting results from field research. You can do this by using the Sociological Abstracts database, which was introduced in Chapter 4 "Beginning a Research Project" . How do the authors describe the strengths and weaknesses of their study? Are any of the strengths or weaknesses described in this section mentioned in the article? Are there additional strengths or weaknesses not mentioned in this section?

Skip to content

Get Revising

Join get revising, already a member.

Field Experiment (Pros and Cons)

• Created by: Evita
• Created on: 18-12-12 10:21

Report Tue 3rd July, 2018 @ 10:30

Similar Psychology resources:

Research Techniques 0.0 / 5

The experimental method 0.0 / 5

types of experiments 0.0 / 5

Psychology Research Methods 0.0 / 5

OCR Psychology G541 Overview 0.0 / 5

Methods in AS Psychology 0.0 / 5

Experiment 0.0 / 5

Psychology- Methods (Experimental) + Others 0.0 / 5

Psychology, psychologists and research methods 3.0 / 5 based on 1 rating

Social Study In Detail - Hofling et al. (1966) 4.0 / 5 based on 2 ratings

Field experiments, explained

Editor’s note: This is part of a series called “The Day Tomorrow Began,” which explores the history of breakthroughs at UChicago.  Learn more here.

A field experiment is a research method that uses some controlled elements of traditional lab experiments, but takes place in natural, real-world settings. This type of experiment can help scientists explore questions like: Why do people vote the way they do? Why do schools fail? Why are certain people hired less often or paid less money?

University of Chicago economists were early pioneers in the modern use of field experiments and conducted innovative research that impacts our everyday lives—from policymaking to marketing to farming and agriculture.  

Jump to a section:

What is a field experiment, why do a field experiment, what are examples of field experiments, when did field experiments become popular in modern economics, what are criticisms of field experiments.

Field experiments bridge the highly controlled lab environment and the messy real world. Social scientists have taken inspiration from traditional medical or physical science lab experiments. In a typical drug trial, for instance, participants are randomly assigned into two groups. The control group gets the placebo—a pill that has no effect. The treatment group will receive the new pill. The scientist can then compare the outcomes for each group.

A field experiment works similarly, just in the setting of real life.

It can be difficult to understand why a person chooses to buy one product over another or how effective a policy is when dozens of variables affect the choices we make each day. “That type of thinking, for centuries, caused economists to believe you can't do field experimentation in economics because the market is really messy,” said Prof. John List, a UChicago economist who has used field experiments to study everything from how people use  Uber and  Lyft to  how to close the achievement gap in Chicago-area schools . “There are a lot of things that are simultaneously moving.”

The key to cleaning up the mess is randomization —or assigning participants randomly to either the control group or the treatment group. “The beauty of randomization is that each group has the same amount of bad stuff, or noise or dirt,” List said. “That gets differenced out if you have large enough samples.”

Though lab experiments are still common in the social sciences, field experiments are now often used by psychologists, sociologists and political scientists. They’ve also become an essential tool in the economist’s toolbox.  

Some issues are too big and too complex to study in a lab or on paper—that’s where field experiments come in.

In a laboratory setting, a researcher wants to control as many variables as possible. These experiments are excellent for testing new medications or measuring brain functions, but they aren’t always great for answering complex questions about attitudes or behavior.

Labs are highly artificial with relatively small sample sizes—it’s difficult to know if results will still apply in the real world. Also, people are aware they are being observed in a lab, which can alter their behavior. This phenomenon, sometimes called the Hawthorne effect, can affect results.

Traditional economics often uses theories or existing data to analyze problems. But, when a researcher wants to study if a policy will be effective or not, field experiments are a useful way to look at how results may play out in real life.

In 2019, UChicago economist Michael Kremer (then at Harvard) was awarded the Nobel Prize alongside Abhijit Banerjee and Esther Duflo of MIT for their groundbreaking work using field experiments to help reduce poverty . In the 1990s and 2000s, Kremer conducted several randomized controlled trials in Kenyan schools testing potential interventions to improve student performance. 

In the 1990s, Kremer worked alongside an NGO to figure out if buying students new textbooks made a difference in academic performance. Half the schools got new textbooks; the other half didn’t. The results were unexpected—textbooks had no impact.

“Things we think are common sense, sometimes they turn out to be right, sometimes they turn out to be wrong,” said Kremer on an episode of  the Big Brains podcast. “And things that we thought would have minimal impact or no impact turn out to have a big impact.”

In the early 2000s, Kremer returned to Kenya to study a school-based deworming program. He and a colleague found that providing deworming pills to all students reduced absenteeism by more than 25%. After the study, the program was scaled nationwide by the Kenyan government. From there it was picked up by multiple Indian states—and then by the Indian national government.

“Experiments are a way to get at causal impact, but they’re also much more than that,” Kremer said in  his Nobel Prize lecture . “They give the researcher a richer sense of context, promote broader collaboration and address specific practical problems.”    

Among many other things, field experiments can be used to:

Study bias and discrimination

A 2004 study published by UChicago economists Marianne Bertrand and Sendhil Mullainathan (then at MIT) examined racial discrimination in the labor market. They sent over 5,000 resumes to real job ads in Chicago and Boston. The resumes were exactly the same in all ways but one—the name at the top. Half the resumes bore white-sounding names like Emily Walsh or Greg Baker. The other half sported African American names like Lakisha Washington or Jamal Jones. The study found that applications with white-sounding names were 50% more likely to receive a callback.

Examine voting behavior

Political scientist Harold Gosnell , PhD 1922, pioneered the use of field experiments to examine voting behavior while at UChicago in the 1920s and ‘30s. In his study “Getting out the vote,” Gosnell sorted 6,000 Chicagoans across 12 districts into groups. One group received voter registration info for the 1924 presidential election and the control group did not. Voter registration jumped substantially among those who received the informational notices. Not only did the study prove that get-out-the-vote mailings could have a substantial effect on voter turnout, but also that field experiments were an effective tool in political science.

Test ways to reduce crime and shape public policy

Researchers at UChicago’s  Crime Lab use field experiments to gather data on crime as well as policies and programs meant to reduce it. For example, Crime Lab director and economist Jens Ludwig co-authored a  2015 study on the effectiveness of the school mentoring program  Becoming a Man . Developed by the non-profit Youth Guidance, Becoming a Man focuses on guiding male students between 7th and 12th grade to help boost school engagement and reduce arrests. In two field experiments, the Crime Lab found that while students participated in the program, total arrests were reduced by 28–35%, violent-crime arrests went down by 45–50% and graduation rates increased by 12–19%.

The earliest field experiments took place—literally—in fields. Starting in the 1800s, European farmers began experimenting with fertilizers to see how they affected crop yields. In the 1920s, two statisticians, Jerzy Neyman and Ronald Fisher, were tasked with assisting with these agricultural experiments. They are credited with identifying randomization as a key element of the method—making sure each plot had the same chance of being treated as the next.

The earliest large-scale field experiments in the U.S. took place in the late 1960s to help evaluate various government programs. Typically, these experiments were used to test minor changes to things like electricity pricing or unemployment programs.

Though field experiments were used in some capacity throughout the 20th century, this method didn’t truly gain popularity in economics until the 2000s. Kremer and List were early pioneers and first began experimenting with the method in the 1990s.

In 2004, List co-authored  a seminal paper defining field experiments and arguing for the importance of the method. In 2008,  he and UChicago economist Steven Levitt published another study tracing the history of field experiments and their impact on economics.

In the past few decades, the use of field experiments has exploded. Today, economists often work alongside NGOs or nonprofit organizations to study the efficacy of programs or policies. They also partner with companies to test products and understand how people use services.  

There are several  ethical discussions happening among scholars as field experiments grow in popularity. Chief among them is the issue of informed consent. All studies that involve human test subjects must be approved by an institutional review board (IRB) to ensure that people are protected.

However, participants in field experiments often don’t know they are in an experiment. While an experiment may be given the stamp of approval in the research community, some argue that taking away peoples’ ability to opt out is inherently unethical. Others advocate for stricter review processes as field experiments continue to evolve.

According to List, another major issue in field experiments is the issue of scale . Many experiments only test small groups—say, dozens to hundreds of people. This may mean the results are not applicable to broader situations. For example, if a scientist runs an experiment at one school and finds their method works there, does that mean it will also work for an entire city? Or an entire country?

List believes that in addition to testing option A and option B, researchers need a third option that accounts for the limitations that come with a larger scale. “Option C is what I call critical scale features. I want you to bring in all of the warts, all of the constraints, whether they're regulatory constraints, or constraints by law,” List said. “Option C is like your reality test, or what I call policy-based evidence.”

This problem isn’t unique to field experiments, but List believes tackling the issue of scale is the next major frontier for a new generation of economists.

Hero photo copyright Shutterstock.com

More Explainers

Dark energy, explained

Improv, Explained

Get more with UChicago News delivered to your inbox.

Recommended Stories

An economist illuminates our giving habits—during the pandemic and…

Collaborating with Kenyan government on development innovations is…

Related Topics

Latest news, uchicago community welcomes new students to campus.

Patrick Jagoda to deliver Aims of Education address Sept. 26

Big Brains podcast

Big Brains podcast: What are we getting wrong about young voters?

Particle physics

Fermilab short-baseline detector detects its first neutrinos

Go 'Inside the Lab' at UChicago

Explore labs through videos and Q&As with UChicago faculty, staff and students

Collapse of bat populations increased infant mortality rate, study finds

AI Insights

AI is biased against speakers of African American English, study finds

Around uchicago.

In Memoriam

Ralph A. Austen, historian of Africa and a ‘scholar’s scholar,’ 1937-2024

Artificial Intelligence

NSF awards $20 million to build AI models that predict scientific discoveries a…

New chicago booth course empowers student entrepreneurs to tackle global issues.

Campus News

Project to improve accessibility, sustainability of Main Quadrangles

UChicago President Paul Alivisatos accepts 2024 Kavli Prize in Nanoscience

Obama Foundation

University of Chicago Obama Foundation Scholars Program includes 18 emerging leaders for 2024-25

Meet A UChicagoan

“The trouble comes from using these immortal materials for disposable products.”

South Side Science Festival

Third annual South Side Science Festival set to bring a day of discovery for all ages on Oct. 5

The Advantages & Disadvantages of Field Experiments in Sociology

Stanley goff.

In his book, “After Virtue,” philosopher Alasdair MacIntyre said that social sciences have consistently lacked predictive power because they are incapable of articulating law-like generalizations in the same way that physical sciences do. The ability of humans to invent, decide and reflexively react, as well as plain chance, makes any field experiment in sociology a risky proposition.

Explore this article

• Field Experiments
• Ethical Issues
• Proving and Disproving Popular Beliefs

1 Field Experiments

A sociological field experiment is any experiment that is carried out beyond the laboratory; that is, the experiment is performed in a “natural setting,” by observation of an unprepared environment, or the experiment is performed in an artificial setting where the variables cannot be controlled. The fact that variables cannot be controlled is the major disadvantage of field experiments from the scientific standpoint. In laboratory experiments, the control of variables makes it possible to establish relations of cause and effect.

2 Advantage

An advantage of field experiments is that the subjects are presumably not as influenced by the observations of the experimenters -- especially if they do not know an experiment is taking place. An experiment sending white and black actors into a similar social situation is one example of subjects who are unaware that an experiment is being conducted. Had they been aware of being subjects, it is unlikely they would have acted naturally -- in this case behaving differently toward black and white actors who otherwise had the same attributes.

3 Ethical Issues

Sociological field experiments present researchers with significant ethical problems. Because field experiments may lack a strong element of control, there is a higher risk of unanticipated actions that can adversely affect subjects and participants. In the case of the Stanford prison experiment, a simulated prison was established to study interactions between students who had volunteered to play the roles of prisoners and guards. Within days, the “guards” had unexpectedly become so abusive that there was serious risk of injury or death to prisoners, and the experiment was shut down. Much of value has been inferred about the behavioral power of roles from this experiment, but there are serious ethical questions about whether researchers have a right to create this kind of risk for subjects.

4 Proving and Disproving Popular Beliefs

Some people might agree or disagree with the statement, "Most people are honest." Some field experiments that can be done simply and at low risk can be used to prove or disprove popular assumptions about social behaviors. A good example is the “honest tea” experiment. A tea company set up unmanned stands in seven cities with bottles of tea for sale at a dollar apiece. Customers were instructed to put a dollar in the money box for a bottle of tea. Hidden cameras recorded that between 75 percent (in Los Angeles) and 93 percent (in Boston) were honest and paid. Obviously, there may have been variables in each city that affected the differences, so no conclusion about the cities are valid; but a general conclusion might be that most people in this situation are honest.

• 1 "After Virtue”; Alasdair Macintyre; 1984
• 2 Stanford Prison Experiment: A Simulation Study of the Psychology of Imprisonment

About the Author

Stanley Goff began writing in 1995. He has published four books: "Hideous Dream," "Full Spectrum Disorder," "Sex & War" and "Energy War," as well as articles, commentary and monographs online. Goff has a Bachelor of Arts in English from the University of the State of New York.

Related Articles

Major Types of Research Design

What Is the Definition of Direct Observation?

Tools Used in Sociology

Good Thesis Statements for "A Tale of Two Cities"

Crime in the Cities in the Early 1900s

Possible Topics for a Research Paper on Criminology

External Vs. Internal Locus of Control

The Characteristics of a Psychopathic Personality

What is the Difference Between Criminal Justice & Criminology?

What Is an Empirical Statement?

Normative Approach to Politics

How to Use the Scientific Method to Research Crime

What Is a Quasi Experimental Study?

Topics for a Business Law Term Paper

The Advantages of a Questionnaire Research Project

Cultural Differences in Moral Reasoning

Reasons for a Hypothesis

Examples of Control & Variable in a Science Experiment

How to Make a Hypothesis Statement

Are Communism & Marxism the Same Thing?

Regardless of how old we are, we never stop learning. Classroom is the educational resource for people of all ages. Whether you’re studying times tables or applying to college, Classroom has the answers.

• Accessibility
• Terms of Use
• Privacy Policy
• Copyright Policy
• Manage Preferences

© 2020 Leaf Group Ltd. / Leaf Group Media, All Rights Reserved. Based on the Word Net lexical database for the English Language. See disclaimer .

Reference Library

Collections

• See what's new
• All Resources
• Student Resources
• Assessment Resources
• Teaching Resources
• CPD Courses
• Livestreams

Study notes, videos, interactive activities and more!

Psychology news, insights and enrichment

Currated collections of free resources

Browse resources by topic

• All Psychology Resources

Resource Selections

Currated lists of resources

Study Notes

Field Experiments

Last updated 22 Mar 2021

• Share on Facebook
• Share on Twitter
• Share by Email

Experiments look for the effect that manipulated variables (independent variables) have on measured variables (dependent variables), i.e. causal effects.

Field experiments are conducted in a natural setting (e.g. at a sports event or on public transport), as opposed to the artificial environment created in laboratory experiments. Some variables cannot be controlled due to the unpredictability of these real-life settings (e.g. the public interacting with participants), but an independent variable will still be altered for a dependent variable to be measured against.

Evaluation of field experiments:

- Field experiments generally yield results with higher ecological validity than laboratory experiments, as the natural settings will relate to real life.

- Demand characteristics are less of an issue with field experiments than laboratory experiments (i.e. participants are less likely to adjust their natural behaviour according to their interpretation of the study’s purpose, as they might not know they are in a study).

- Extraneous variables could confound results due to the reduced control experimenters have over them in non-artificial environments, which makes it difficult to find truly causal effects between independent and dependent variables.

- Ethical principles have to be considered, such as the lack of informed consent; if participants are not made aware of their participation in an experiment, privacy must be respected during observations and participants must be debriefed appropriately when observations come to an end.

- Precise replication of the natural environment of field experiments is understandably difficult, so they have poor reliability, unlike laboratory experiments where the exact conditions can be recreated.

- Field experiments are more susceptible to sample bias, as participants are often not randomly allocated to experimental conditions (i.e. participants’ groups are already pre-set rather than randomly assigned).

• Field experiments

You might also like

Types of experiment: overview, our subjects.

• › Criminology
• › Economics
• › Geography
• › Health & Social Care
• › Psychology
• › Sociology
• › Teaching & learning resources
• › Student revision workshops
• › Online student courses
• › CPD for teachers
• › Livestreams
• › Teaching jobs

Boston House, 214 High Street, Boston Spa, West Yorkshire, LS23 6AD Tel: 01937 848885

• › Contact us
• › Terms of use
• › Privacy & cookies

© 2002-2024 Tutor2u Limited. Company Reg no: 04489574. VAT reg no 816865400.

• Books, Journals, Papers
• Guides & How To’s
• Life Around The World
• Research Methods
• Functionalism
• Postmodernism
• Social Constructionism
• Structuralism
• Symbolic Interactionism
• Sociology Theorists
• General Sociology
• Social Policy
• Social Work
• Sociology of Childhood
• Sociology of Crime & Deviance
• Sociology of Art
• Sociology of Dance
• Sociology of Food
• Sociology of Sport
• Sociology of Disability
• Sociology of Economics
• Sociology of Education
• Sociology of Emotion
• Sociology of Family & Relationships
• Sociology of Gender
• Sociology of Health
• Sociology of Identity
• Sociology of Ideology
• Sociology of Inequalities
• Sociology of Knowledge
• Sociology of Language
• Sociology of Law
• Sociology of Anime
• Sociology of Film
• Sociology of Gaming
• Sociology of Literature
• Sociology of Music
• Sociology of TV
• Sociology of Migration
• Sociology of Nature & Environment
• Sociology of Politics
• Sociology of Power
• Sociology of Race & Ethnicity
• Sociology of Religion
• Sociology of Sexuality
• Sociology of Social Movements
• Sociology of Technology
• Sociology of the Life Course
• Sociology of Travel & Tourism
• Sociology of Violence & Conflict
• Sociology of Work
• Urban Sociology
• Changing Relationships Within Families
• Conjugal Role Relationships
• Criticisms of Families
• Family Forms
• Functions of the Family
• Featured Articles
• Privacy Policy
• Terms & Conditions

Field Experiment in Sociology: Concept, Methodology, and Applications

Field experiments are a critical methodological approach in sociology that allows researchers to study social phenomena in natural settings while maintaining control over the experimental conditions. This method bridges the gap between the control typically afforded by laboratory experiments and the realism of observational studies. Field experiments are particularly valuable in sociology for testing theories about social behavior and interactions in real-world environments. This essay outlines the concept of a field experiment, discusses its methodology, and explores its applications and significance in the field of sociology.

Conceptual Framework

A field experiment involves the manipulation of variables in a natural setting to observe the effects on a specific outcome. Unlike laboratory experiments, field experiments are conducted in the environments where participants naturally occur, which may include places like schools, workplaces, or public areas. This setting ensures that the behavior observed is more likely to reflect real-world behaviors, providing insights that are often more generalizable.

Methodology of Field Experiments

The methodology of conducting a field experiment in sociology involves several key steps:

• Hypothesis Formation : The process begins with the formulation of a hypothesis based on existing theories or preliminary observations. This hypothesis predicts how changes in an in dependent variable (manipulated by the researcher) will affect a dependent variable.
• Randomized Controlled Trials (RCTs) : Participants are randomly assigned to either the treatment or control group to ensure that any differences observed can be attributed to the intervention itself.
• Before-and-After Studies : Measurements are taken both before and after the intervention on the same participants to see changes over time.
• Quasi-Experimental Designs : These are used when random assignment is not possible. Researchers rely on existing natural divisions in the study population.
• Selection of Participants : Participants are selected to represent the broader population. In some cases, the entire setting (like a specific community or organization) becomes part of the experiment.
• Implementation of the Treatment : The independent variable is manipulated as per the experimental design. For example, introducing a new educational curriculum in a school to see its effect on student performance.
• Data Collection and Analysis : Data are collected on the outcomes of interest. Statistical methods are then used to analyze the data to determine if the changes in the independent variable have had a statistically significant effect on the dependent variable.
• Ethical Considerations : Field experiments must adhere to strict ethical guidelines, particularly regarding consent, potential harm, and the privacy of participants.

Applications in Sociological Research

Field experiments have a broad range of applications in sociology, including:

• Social Norms and Compliance : Researchers might manipulate environmental cues or norms to see how they affect compliance with social behaviors, such as recycling or adherence to public policies.
• Education and Socialization : Field experiments can test the effects of different educational interventions on student outcomes, helping to identify effective teaching practices.
• Labor Market Dynamics : Interventions such as resume modification can be used to study discrimination in hiring practices.
• Health and Social Policies : Introducing health interventions in a community and observing changes in health behaviors or outcomes.

Strengths and Limitations

Field experiments offer several strengths, including high external validity due to the naturalistic setting, the ability to infer causality by controlling for confounding variables, and the practical application of sociological theories. However, they also face limitations:

• Control over Variables : While offering more control than natural observations, field experiments cannot control for all external variables like a lab experiment.
• Ethical and Practical Challenges : Manipulating environments and behaviors can raise ethical concerns, and obtaining genuine consent in natural settings can be complex.
• Cost and Complexity : Field experiments can be expensive and logistically complex to design and execute, especially in larger settings.

Field experiments are a dynamic and robust methodological approach in sociology that allows researchers to study complex social phenomena in natural settings. By carefully designing and conducting these experiments, sociologists can provide valuable insights into human behavior and social structures, ultimately contributing to more effective policies and interventions. Despite their challenges, the depth and applicability of the data derived from field experiments make them an indispensable tool in the sociologist’s toolkit. As social contexts and technologies evolve, the methods and applications of field experiments in sociology will continue to expand, offering even greater opportunities for discovery and social improvement.

Mr Edwards has a PhD in sociology and 10 years of experience in sociological knowledge

Related Articles

Explanandum and Explanans in Sociology

In sociological research and theory, the concepts of 'explanandum' and 'explanans' are pivotal in the structuring of explanations. These terms...

Understanding Documentary Research in Sociology

Learn about documentary research in sociology, a methodological approach that involves analyzing existing documents to gain insights into social phenomena....

Ethnography: A Comprehensive Sociological Approach

A Silent Voice: A Sociological Analysis

Participate in our sociology course poll, get the latest sociology.

How would you rate the content on Easy Sociology?

Recommended

Understanding Tax Avoidance through a Sociological Lens

Functionalism: An Introduction

24 hour trending.

Robert Merton’s Strain Theory Explained

Age stratification: understanding social hierarchies based on age, understanding conflict theories in sociology, the work and contributions of emile durkheim in sociology, the marxist view of crime and deviance: an outline, explanation, and analysis.

Easy Sociology makes sociology as easy as possible. Our aim is to make sociology accessible for everybody. © 2023 Easy Sociology

© 2023 Easy Sociology

Experiments in Sociology – Revision Notes

Table of Contents

Last Updated on September 6, 2021 by

Definitions, key features and the theoretical, practical and ethical strengths and limitations of laboratory and field experiments applied to sociology (and psychology). Also covers key terms related to experiments.

post has been written to help students revising for the research methods aspect of their second year A-level exams.

Experiments – The Basics: Definitions/ Key Features

Advantages of laboratory experiments, disadvantages of laboratory experiments, advantages of field experiments over laboratory experiments, the relative disadvantages of field experiments, experiments – key terms summary.

Dependent Variable – this is the object of the study in the experiment, the variable which will (possibly) be effected by the independent variables.

Extraneous variables – Variables which are not of interest to the researcher but which may interfere with the results of an experiment

Experimental group – The group under study in the investigation.

Signposting

This post has been written to help students revising for the research methods aspect of their second year A-level exams.

Laboratory experiments in sociology

Share this:

One thought on “experiments in sociology – revision notes”, leave a reply cancel reply, discover more from revisesociology.

Experimental Method In Psychology

Saul McLeod, PhD

Editor-in-Chief for Simply Psychology

BSc (Hons) Psychology, MRes, PhD, University of Manchester

Saul McLeod, PhD., is a qualified psychology teacher with over 18 years of experience in further and higher education. He has been published in peer-reviewed journals, including the Journal of Clinical Psychology.

Learn about our Editorial Process

Olivia Guy-Evans, MSc

Associate Editor for Simply Psychology

BSc (Hons) Psychology, MSc Psychology of Education

Olivia Guy-Evans is a writer and associate editor for Simply Psychology. She has previously worked in healthcare and educational sectors.

On This Page:

The experimental method involves the manipulation of variables to establish cause-and-effect relationships. The key features are controlled methods and the random allocation of participants into controlled and experimental groups .

What is an Experiment?

An experiment is an investigation in which a hypothesis is scientifically tested. An independent variable (the cause) is manipulated in an experiment, and the dependent variable (the effect) is measured; any extraneous variables are controlled.

An advantage is that experiments should be objective. The researcher’s views and opinions should not affect a study’s results. This is good as it makes the data more valid  and less biased.

There are three types of experiments you need to know:

1. Lab Experiment

A laboratory experiment in psychology is a research method in which the experimenter manipulates one or more independent variables and measures the effects on the dependent variable under controlled conditions.

A laboratory experiment is conducted under highly controlled conditions (not necessarily a laboratory) where accurate measurements are possible.

The researcher uses a standardized procedure to determine where the experiment will take place, at what time, with which participants, and in what circumstances.

Participants are randomly allocated to each independent variable group.

Examples are Milgram’s experiment on obedience and  Loftus and Palmer’s car crash study .

• Strength : It is easier to replicate (i.e., copy) a laboratory experiment. This is because a standardized procedure is used.
• Strength : They allow for precise control of extraneous and independent variables. This allows a cause-and-effect relationship to be established.
• Limitation : The artificiality of the setting may produce unnatural behavior that does not reflect real life, i.e., low ecological validity. This means it would not be possible to generalize the findings to a real-life setting.
• Limitation : Demand characteristics or experimenter effects may bias the results and become confounding variables .

2. Field Experiment

A field experiment is a research method in psychology that takes place in a natural, real-world setting. It is similar to a laboratory experiment in that the experimenter manipulates one or more independent variables and measures the effects on the dependent variable.

However, in a field experiment, the participants are unaware they are being studied, and the experimenter has less control over the extraneous variables .

Field experiments are often used to study social phenomena, such as altruism, obedience, and persuasion. They are also used to test the effectiveness of interventions in real-world settings, such as educational programs and public health campaigns.

An example is Holfing’s hospital study on obedience .

• Strength : behavior in a field experiment is more likely to reflect real life because of its natural setting, i.e., higher ecological validity than a lab experiment.
• Strength : Demand characteristics are less likely to affect the results, as participants may not know they are being studied. This occurs when the study is covert.
• Limitation : There is less control over extraneous variables that might bias the results. This makes it difficult for another researcher to replicate the study in exactly the same way.

3. Natural Experiment

A natural experiment in psychology is a research method in which the experimenter observes the effects of a naturally occurring event or situation on the dependent variable without manipulating any variables.

Natural experiments are conducted in the day (i.e., real life) environment of the participants, but here, the experimenter has no control over the independent variable as it occurs naturally in real life.

Natural experiments are often used to study psychological phenomena that would be difficult or unethical to study in a laboratory setting, such as the effects of natural disasters, policy changes, or social movements.

For example, Hodges and Tizard’s attachment research (1989) compared the long-term development of children who have been adopted, fostered, or returned to their mothers with a control group of children who had spent all their lives in their biological families.

Here is a fictional example of a natural experiment in psychology:

Researchers might compare academic achievement rates among students born before and after a major policy change that increased funding for education.

In this case, the independent variable is the timing of the policy change, and the dependent variable is academic achievement. The researchers would not be able to manipulate the independent variable, but they could observe its effects on the dependent variable.

• Strength : behavior in a natural experiment is more likely to reflect real life because of its natural setting, i.e., very high ecological validity.
• Strength : Demand characteristics are less likely to affect the results, as participants may not know they are being studied.
• Strength : It can be used in situations in which it would be ethically unacceptable to manipulate the independent variable, e.g., researching stress .
• Limitation : They may be more expensive and time-consuming than lab experiments.
• Limitation : There is no control over extraneous variables that might bias the results. This makes it difficult for another researcher to replicate the study in exactly the same way.

Key Terminology

Ecological validity.

The degree to which an investigation represents real-life experiences.

Experimenter effects

These are the ways that the experimenter can accidentally influence the participant through their appearance or behavior.

Demand characteristics

The clues in an experiment lead the participants to think they know what the researcher is looking for (e.g., the experimenter’s body language).

Independent variable (IV)

The variable the experimenter manipulates (i.e., changes) is assumed to have a direct effect on the dependent variable.

Dependent variable (DV)

Variable the experimenter measures. This is the outcome (i.e., the result) of a study.

Extraneous variables (EV)

All variables which are not independent variables but could affect the results (DV) of the experiment. EVs should be controlled where possible.

Confounding variables

Variable(s) that have affected the results (DV), apart from the IV. A confounding variable could be an extraneous variable that has not been controlled.

Random Allocation

Randomly allocating participants to independent variable conditions means that all participants should have an equal chance of participating in each condition.

The principle of random allocation is to avoid bias in how the experiment is carried out and limit the effects of participant variables.

Order effects

Changes in participants’ performance due to their repeating the same or similar test more than once. Examples of order effects include:

(i) practice effect: an improvement in performance on a task due to repetition, for example, because of familiarity with the task;

(ii) fatigue effect: a decrease in performance of a task due to repetition, for example, because of boredom or tiredness.

• Previous Chapter
• Next Chapter

Over the past two decades, experimental economics has seen a large increase in the use of field experiments. Field experiments provide a useful bridge between the stylized environment of the laboratory and the context-rich environment of the outside world. Field experiments have now been used in a range of applications, including in development economics, charitable giving, labor economics, discrimination in markets, financial decision-making, education and health. When comparing the laboratory and field experiments, two themes emerge. The first is whether results from the experiment can be generalized, and the second is the level of control that the experimenter has. These themes represent a trade-off, since generalizability can sometimes be higher in the field, but this comes with some loss of control. This chapter provides an overview of field experiments, explores the advantages and disadvantages of the method, and provides a how-to guide for new researchers to the method.

You are not authenticated to view the full text of this chapter or article.

Access options

Get access to the full article by using one of the access options below.

Other access options

Institutional Login

Log in with Open Athens, Shibboleth, or your institutional credentials

Personal login

Log in with your Elgar Online account

Table of Contents

• [185.66.14.133]
• 185.66.14.133

Character limit 500 /500

Learning Materials

• Business Studies
• Combined Science
• Computer Science
• Engineering
• English Literature
• Environmental Science
• Human Geography
• Macroeconomics
• Microeconomics
• Field Experiment

Sometimes, a laboratory setting isn't the best option for investigating a phenomenon when conducting research. Whilst lab experiments offer a lot of control, they are artificial and do not truly represent the real world, which causes issues with ecological validity. This is where field experiments come in. 

Millions of flashcards designed to help you ace your studies

• Cell Biology

Which of the following statement is correct? 

What type of validity is mundane realism related to? 

What are the potential ethical issues that may happen when doing a field experiment? 

True or false: A potentially biased sample is an issue for natural experiments. 

True or false: In field experiments, variables are manipulated. 

Can variables be manipulated in real-life settings? 

A study aimed to explore how early parental interactions influenced later attachment styles. Would it be appropriate to use a field experiment? 

Researchers have a higher level of control in           experiments compared to               experiments.

A study aimed to explore how children responded to authoritative figures versus non-authoritative ones at school. Could a field experiment be used? 

Can a field experiment be used to compare children's behaviour around their usual and substitute teachers? 

Review generated flashcards

to start learning or create your own AI flashcards

Start learning or create your own AI flashcards

Vaia Editorial Team

Team Field Experiment Teachers

• 5 minutes reading time
• Checked by Vaia Editorial Team
• Approaches in Psychology
• Basic Psychology
• Biological Bases of Behavior
• Biopsychology
• Careers in Psychology
• Clinical Psychology
• Cognition and Development
• Cognitive Psychology
• Data Handling and Analysis
• Developmental Psychology
• Eating Behaviour
• Emotion and Motivation
• Famous Psychologists
• Forensic Psychology
• Health Psychology
• Individual Differences Psychology
• Issues and Debates in Psychology
• Personality in Psychology
• Psychological Treatment
• Relationships
• Research Methods in Psychology
• Aims and Hypotheses
• Causation in Psychology
• Coding Frame Psychology
• Correlational Studies
• Cross Cultural Research
• Cross Sectional Research
• Ethical Issues and Ways of Dealing with Them
• Experimental Designs
• Features of Science
• Independent Group Design
• Lab Experiment
• Longitudinal Research
• Matched Pairs Design
• Meta Analysis
• Natural Experiment
• Observational Design
• Online Research
• Paradigms and Falsifiability
• Peer Review and Economic Applications of Research
• Pilot Studies and the Aims of Piloting
• Quality Criteria
• Questionnaire Construction
• Repeated Measures Design
• Research Methods
• Sampling Frames
• Sampling Psychology
• Scientific Processes
• Scientific Report
• Scientific Research
• Self-Report Design
• Self-Report Techniques
• Semantic Differential Rating Scale
• Snowball Sampling
• Schizophrenia
• Scientific Foundations of Psychology
• Scientific Investigation
• Sensation and Perception
• Social Context of Behaviour
• Social Psychology

Jump to a key chapter

Despite its name, field experiments, whilst they can be conducted in a field, are not restricted to a literal field.

Both laboratory and field experiments manipulate a variable to see if it can be controlled and affect the dependent variable. Also, both are valid forms of experimentation.

• We will start by learning the field experiment definition and identify how field experiments are used in research.
• Moving on from this, we will explore a field experiment example carried out by Hofling in 1966.
• Finally, we will discuss the field experiment's advantages and disadvantages.

Field Experiment Definition

A field experiment is a research method where the independent variable is manipulated, and the dependent variable is measured in a real-world setting.

If you had to research travel, a field experiment could be performed on a train. Also, you could analyse a car or bike ride out in the streets. Similarly, someone might conduct an experiment in a school investigating different phenomena present in classrooms or school playgrounds.

Field Experiment: Psychology

Field experiments are usually designed and used in psychology when researchers want to observe participants in their natural environment, but the phenomenon is not naturally occurring. Therefore, the researcher must manipulate the investigated variables to measure the outcome, e.g. how students behaviour when a teacher or a substitute teacher is present.

The procedure of field experiments in psychology is the following:

• Identify a research question, variables , and hypotheses.
• Recruit participants.
• Carry out the investigation.
• Analyse data and report results.

Field Experiment: Example

Hofling (1966) conducted a field experiment to investigate obedience in nurses. The study recruited 22 nurses working in a psychiatric hospital on a night shift, although they were unaware they were taking part in the study.

D uring their shift, a doctor, who was actually the researcher, called the nurses and asked them to urgently administer 20mg of a drug to a patient (double the maximum dosage). The doctor/researcher told the nurses that he would authorise medication administration later.

The research aimed to identify if people broke the rules and obeyed authoritative figures' orders.

The results showed that 95% of the nurses obeyed the order, despite breaking the rules. Only one questioned the doctor.

The Hofling study is an example of a field experiment. It was carried out in a natural setting, and the researcher manipulated the situation (instructed nurses to administer high-dosage medication) to see if it affected whether nurses obeyed the authoritative figure or not.

Field Experiment: Advantages and Disadvantages

Like any type of research, field experiments have certain advantages and disadvantages that must be considered before opting for this research method.

Field Experiments: Advantages

Some of the advantages of field experiments include the following:

• The results are more likely to reflect real-life compared to laboratory research, as they have higher ecological validity.

Hawthorne effect is when people adjust their behaviour because they know they are being observed.

• It is high in mundane realism compared to lab research; this refers to the extent to which the setting and materials used in a study reflect real-life situations. Field experiments have high mundane realism. Thus, they have high external validity.

A field experiment would be an appropriate research design when investigating children's behaviour changes at school. More specifically, to compare their behaviours around their usual and substitute teachers.

• It can establish c ausal relationships because researchers manipulate a variable and measure its effect. However, extraneous variables can make this difficult. We will address these issues in the next paragraph.

Field Experiments: Disadvantages

The disadvantages of field experiments are the following:

• Researchers have less control over extraneous/confounding variables, reducing confidence in establishing causal relationships .
• It is difficult to replicate the research, making it hard to determine the results' reliability.
• This experimental method has a high chance of collecting a biased sample, making it difficult to generalise the results.
• It may not be easy to record data accurately with so many variables present. Overall, field experiments have less control.
• Potential ethical issues of field experiments include: difficulty getting informed consent, and the researcher may need to deceive participants.

Field Experiment - Key Takeaways

• The field experiment definition is a research method where the independent variable is manipulated, and the dependent variable is measured in a real-world setting.
• Field experiments are usually used in psychology when researchers want to observe participants in their natural environment. The phenomenon is not naturally occurring, so the researcher must manipulate the variables to measure the outcome.
• Hofling (1966) used a field experiment to investigate if nurses wrongfully obeyed authoritative figures in their workplace.
• Field experiments have high ecological validity, establish causal relationships, and reduce the chances of demand characteristics interfering with research.
• However, they offer less control, and confounding variables may be an issue. From the ethical perspective, participants cannot always consent to participate and may need to be deceived to be observed. Replicating field experiments is also difficult.

Flashcards in Field Experiment 10

The researcher manipulates the variables in a controlled setting.

External validity.

Deception of participants.

Learn with 10 Field Experiment flashcards in the free Vaia app

We have 14,000 flashcards about Dynamic Landscapes.

Already have an account? Log in

Frequently Asked Questions about Field Experiment

What is a field experiment? 

A field experiment is a research method where the independent variable is manipulated, and the dependent variable is measured in a real-world setting. 

What is the difference between natural and field experiments?

In field experiments, researchers manipulate the independent variable. On the other hand, in natural experiments, the researcher does not manipulate anything in the investigation. 

What is an example of a field experiment? 

Hofling (1966) utilised a field experiment to identify if nurses would break the rules and obey an authoritative figure. 

What is one drawback of field experiments? 

A disadvantage of a field experiment is that researchers cannot control the extraneous variables, and this may reduce the validity of the findings. 

How to conduct a field experiment?

The steps for conducting a field experiment are: 

• identify a research question, variables, and hypotheses 
• recruit participants
• carry out the experiment
• analyse the data and report the results 

Test your knowledge with multiple choice flashcards

Join the Vaia App and learn efficiently with millions of flashcards and more!

Keep learning, you are doing great.

Discover learning materials with the free Vaia app

Vaia is a globally recognized educational technology company, offering a holistic learning platform designed for students of all ages and educational levels. Our platform provides learning support for a wide range of subjects, including STEM, Social Sciences, and Languages and also helps students to successfully master various tests and exams worldwide, such as GCSE, A Level, SAT, ACT, Abitur, and more. We offer an extensive library of learning materials, including interactive flashcards, comprehensive textbook solutions, and detailed explanations. The cutting-edge technology and tools we provide help students create their own learning materials. StudySmarter’s content is not only expert-verified but also regularly updated to ensure accuracy and relevance.

Team Psychology Teachers

Study anywhere. Anytime.Across all devices.

Create a free account to save this explanation..

Save explanations to your personalised space and access them anytime, anywhere!

By signing up, you agree to the Terms and Conditions and the Privacy Policy of Vaia.

Sign up to highlight and take notes. It’s 100% free.

Join over 22 million students in learning with our Vaia App

The first learning app that truly has everything you need to ace your exams in one place

• Flashcards & Quizzes
• AI Study Assistant
• Study Planner
• Smart Note-Taking

Privacy Overview

Our systems are now restored following recent technical disruption, and we’re working hard to catch up on publishing. We apologise for the inconvenience caused. Find out more: https://www.cambridge.org/universitypress/about-us/news-and-blogs/cambridge-university-press-publishing-update-following-technical-disruption

We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings .

Login Alert

• > Cambridge Handbook of Routine Dynamics
• > Field Experiments and Routine Dynamics

Book contents

• Cambridge Handbook of Routine Dynamics
• Copyright page
• Contributors
• Chapter 1 What Is Routine Dynamics?
• Part I Theoretical Resources for Routine Dynamics Research
• Part II Methodological Issues in Routine Dynamics Research
• Chapter 8 Ethnography and Routine Dynamics
• Chapter 9 Video Methods and Routine Dynamics
• Chapter 10 Field Experiments and Routine Dynamics
• Chapter 11 Agent-Based Modelling in Routine Dynamics
• Chapter 12 Sequence Analysis in Routine Dynamics
• Chapter 13 Narrative Networks in Routine Dynamics
• Chapter 14 Bakhtin’s Chronotope and Routine Dynamics
• Part III Themes in Routine Dynamics Research
• Part IV Related Communities of Thought
• Author Index
• Subject Index

Chapter 10 - Field Experiments and Routine Dynamics

from Part II - Methodological Issues in Routine Dynamics Research

Published online by Cambridge University Press:  11 December 2021

Experimental approaches are gaining in popularity across disciplines, ranging from behavioural sciences to economics. In this chapter, we discuss the advantages and disadvantages of field experiments and review their use by scholars to study routine dynamics. Based on these, we suggest that field experiments hold further promise to study routines given their potential to develop and test theory, while achieving internal and external validity. To further the adoption of field experiments to study routines, we outline a five-step procedure, including research questions and hypotheses, context and research setting, treatment and design, measurement and statistical tests, and managing field experiments. We conclude by discussing potential research questions and contexts suitable for field experiments.

Access options

Save book to kindle.

To save this book to your Kindle, first ensure [email protected] is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle .

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service .

• Field Experiments and Routine Dynamics
• By Hari Bapuji , Manpreet Hora , Huashan Li
• Edited by Martha S. Feldman , University of California, Irvine , Brian T. Pentland , Michigan State University , Luciana D'Adderio , University of Edinburgh , Katharina Dittrich , University of Warwick , Claus Rerup , David Seidl
• Book: Cambridge Handbook of Routine Dynamics
• Online publication: 11 December 2021
• Chapter DOI: https://doi.org/10.1017/9781108993340.013

Save book to Dropbox

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Dropbox .

Save book to Google Drive

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Google Drive .

• A-Z Publications

Annual Review of Sociology

Volume 43, 2017, review article, field experiments across the social sciences.

• Delia Baldassarri 1 , and Maria Abascal 2
• View Affiliations Hide Affiliations Affiliations: 1 Department of Sociology, New York University, New York, New York 10012; email: [email protected] 2 Department of Sociology, Columbia University, New York, New York 10027; email: [email protected]
• Vol. 43:41-73 (Volume publication date July 2017) https://doi.org/10.1146/annurev-soc-073014-112445
• First published as a Review in Advance on May 22, 2017
• © Annual Reviews

Using field experiments, scholars can identify causal effects via randomization while studying people and groups in their naturally occurring contexts. In light of renewed interest in field experimental methods, this review covers a wide range of field experiments from across the social sciences, with an eye to those that adopt virtuous practices, including unobtrusive measurement, naturalistic interventions, attention to realistic outcomes and consequential behaviors, and application to diverse samples and settings. The review covers four broad research areas of substantive and policy interest: first, randomized controlled trials, with a focus on policy interventions in economic development, poverty reduction, and education; second, experiments on the role that norms, motivations, and incentives play in shaping behavior; third, experiments on political mobilization, social influence, and institutional effects; and fourth, experiments on prejudice and discrimination. We discuss methodological issues concerning generalizability and scalability as well as ethical issues related to field experimental methods. We conclude by arguing that field experiments are well equipped to advance the kind of middle-range theorizing that sociologists value.

Article metrics loading...

Full text loading...

Literature Cited

• Abascal M . 2015 . Us and them: black–white relations in the wake of Hispanic population growth. Am. Sociol. Rev. 80 : 789– 813 [Google Scholar]
• Adida CL , Laitin DD , Valfort MA . 2016 . Why Muslim Integration Fails in Christian-Heritage Societies Cambridge, MA: Harvard Univ. Press [Google Scholar]
• Ahmed AM , Hammarstedt M . 2008 . Discrimination in the rental housing market: a field experiment on the Internet. J. Urban Econ. 64 : 362– 72 [Google Scholar]
• Ahmed AM , Hammarstedt M . 2009 . Detecting discrimination against homosexuals: evidence from a field experiment on the Internet. Economica 76 : 599– 97 [Google Scholar]
• Arceneaux K , Nickerson DW . 2009 . Who is mobilized to vote? A re-analysis of 11 field experiments. Am. J. Political Sci. 53 : 1– 16 [Google Scholar]
• Attanasio O , Augsburg B , De Haas R , Fitzsimons E , Harmgart H . 2012 . Group lending or individual lending? Evidence from a randomised field experiment in Mongolia. Work. Pap. No. 136, Eur. Bank Reconstr. Dev. [Google Scholar]
• Attanasio O , Pellerano L , Reyes SP . 2009 . Building trust? Conditional cash transfer programmes and social capital. Fiscal Stud. 30 : 139– 77 [Google Scholar]
• Avdeenko A , Gilligan MG . 2015 . International interventions to build social capital: evidence from a field experiment in Sudan. Am. Political Sci. Rev. 109 : 427– 49 [Google Scholar]
• Ayres I , Siegelman P . 1995 . Race and gender discrimination in bargaining for a new car. Am. Econ. Rev. 85 : 304– 21 [Google Scholar]
• Baldassarri D . 2015 . Cooperative networks: altruism, group solidarity, and reciprocity in Ugandan farmer organizations. Am. J. Sociol. 121 : 355– 95 [Google Scholar]
• Baldassarri D . 2016 . Prosocial behavior across communities: evidence from a nationwide lost-letter experiment Presented at Advances with Field Experiments Conf., Sept. 16, Univ Chicago: [Google Scholar]
• Banerjee A , Bertrand M , Datta S , Mullainathan S . 2009 . Labor market discrimination in Delhi: evidence from a field experiment. J. Comp. Econ. 37 : 14– 27 [Google Scholar]
• Banerjee A , Duflo E . 2009 . The experimental approach to development economics. Annu. Rev. Econ. 1 : 151– 78 [Google Scholar]
• Banerjee A , Duflo E . 2011 . Poor Economics: A Radical Rethinking of the Way to Fight Global Poverty. New York: Public Affairs [Google Scholar]
• Banerjee A , Duflo E , Glennerster R , Kothari D . 2010a . Improving immunization coverage in rural India: Clustered randomized controlled immunisation campaigns with and without incentives. Br. Med. J. 340:c2220 [Google Scholar]
• Banerjee A , Duflo E , Glennerster R , Kinnan C . 2010b . The miracle of microfinance? Evidence from a randomized evaluation. Work. Pap. No. 13-09, Dep. Econ., MIT [Google Scholar]
• Barr A . 2003 . Trust and expected trustworthiness: experimental evidence from Zimbabwean villages. Econ. J. 113 : 614– 30 [Google Scholar]
• Bauchet J , Marshall C , Starita L , Thomas J , Yalouris A . 2011 . Latest findings from randomized evaluations of microfinance. Access Finance Forum Rep. 2 : 1– 27 [Google Scholar]
• Beath A , Christia F , Enikolopov R . 2013 . Empowering women: evidence from a field experiment in Afghanistan. Am. Political Sci. Rev. 107 : 540– 57 [Google Scholar]
• Benson PL , Karabenick SA , Lerner RM . 1976 . Pretty pleases: the effects of physical attractiveness, race, and sex on receiving help. J. Exp. Soc. Psychol. 12 : 409– 15 [Google Scholar]
• Benz M , Meier S . 2008 . Do people behave in experiments as in the field? Evidence from donations. Exp. Econ. 11 : 278– 81 [Google Scholar]
• Bertrand M , Karlan D , Mullainathan S , Shafir E , Zinman J . 2010 . What's advertising content worth? Evidence from a consumer credit marketing field experiment. Q. J. Econ. 125 : 263– 306 [Google Scholar]
• Bertrand M , Mullainathan S . 2004 . Are Emily and Greg more employable than Lakisha and Jamal? A field experiment on labor market discrimination. Am. Econ. Rev. 94 : 991– 1013 [Google Scholar]
• Besbris M , Faber JW , Rich P , Sharkey P . 2015 . Effect of neighborhood stigma on economic transitions. PNAS 112 : 4994– 98 [Google Scholar]
• Bettinger EP . 2012 . Paying to learn: the effect of financial incentives on elementary school test scores. Rev. Econ. Stat. 94 : 686– 98 [Google Scholar]
• Bigoni M , Bortolotti S , Casari M , Gambetta D , Pancotto F . 2016 . Amoral familism, social capital, or trust? The behavioural foundations of the Italian north–south divide. Econ. J. 126 : 1318– 41 [Google Scholar]
• Blommaert L , Coenders M , van Tubergen F . 2014 . Discrimination of Arabic-named applicants in the Netherlands: an Internet-based field experiment examining different phases in online recruitment procedures. Soc. Forces 92 : 957– 82 [Google Scholar]
• Bond RM , Fariss CJ , Jones JJ , Kramer AD , Marlow C . et al. 2012 . A 61-million-person experiment in social influence and political mobilization. Nature 489 : 295– 98 [Google Scholar]
• Bosch M , Carnero MA , Farré L . 2010 . Information and discrimination in the rental housing market: evidence from a field experiment. Reg. Sci. Urban Econ. 40 : 11– 19 [Google Scholar]
• Brearley HC . 1931 . Experimental sociology in the United States. Soc. Forces 10 : 196– 99 [Google Scholar]
• Butler DM , Broockman DE . 2011 . Do politicians racially discriminate against constituents? A field experiment on state legislators. Am. J. Political Sci. 55 : 463– 77 [Google Scholar]
• Butler DM , Nickerson DW . 2011 . Can learning constituency opinion affect how legislators vote? Results from a field experiment. Q. J. Political Sci. 6 : 55– 83 [Google Scholar]
• Camerer C . 2003 . Behavioral Game Theory: Experiments in Strategic Interaction New York, NY: Russell Sage Found. [Google Scholar]
• Cardenas J , Carpenter J . 2008 . Behavioural development economics: lessons from field labs in the developing world. J. Dev. Stud. 44 : 337– 64 [Google Scholar]
• Casey K , Glennerster R , Miguel E . 2012 . Reshaping institutions: evidence on external aid and local collective action. Q. J. Econ. 127 : 1755– 812 [Google Scholar]
• Castilla EJ , Benard S . 2010 . The paradox of meritocracy in organizations. Adm. Sci. Q. 55 : 543– 676 [Google Scholar]
• Centola D . 2010 . The spread of behavior in an online social network experiment. Science 329 : 1194– 97 [Google Scholar]
• Charness G , Gneezy U . 2009 . Incentives to exercise. Econometrica 77 : 909– 31 [Google Scholar]
• Chetty R , Hendren N , Katz LF . 2015 . The effects of exposure to better neighborhoods on children: new evidence from the moving to opportunity experiment. Work. Pap. 21156, NBER, Cambridge, MA [Google Scholar]
• Chong D , Junn J . 2011 . Politics from the perspective of minority populations. Cambridge Handbook of Experimental Political Science JN Druckman, DP Green, JH Kuklinski, A Lupia, 602– 33 Cambridge, UK: Cambridge Univ. Press [Google Scholar]
• Cialdini RB , Ascani K . 1976 . Test of a concession procedure for inducing verbal, behavioral, and further compliance with a request to give blood. J. Pers. Soc. Psychol. 61 : 295– 300 [Google Scholar]
• Cialdini RB , Vincent JE , Lewis SK , Catalan J , Wheeler D , Darby BL . 1975 . Reciprocal concessions procedure for inducing compliance: the door-in-the-face technique. J. Pers. Soc. Psychol. 31 : 206– 15 [Google Scholar]
• Clampet-Lundquist S , Massey DS . 2008 . Neighborhood effects on economic self-sufficiency: a reconsideration of the Moving to Opportunity experiment. Am. J. Sociol. 114 : 107– 43 [Google Scholar]
• Cohen J , Dupas P . 2010 . Free distribution or cost-sharing? Evidence from a randomized malaria prevention experiment. Q. J. Econ. 125 : 1– 40 [Google Scholar]
• Cole S , Giné X , Tobacman J , Topalova P , Townsend R , Vickery J . 2013 . Barriers to household risk management: evidence from India. Am. Econ. J. Appl. Econ. 5 : 104– 35 [Google Scholar]
• Cook TD , Shadish WR . 1994 . Social experiments: some developments over the past fifteen years. Annu. Rev. Psychol. 45 : 545– 80 [Google Scholar]
• Correll SJ , Benard S , Paik I . 2007 . Getting a job: is there a motherhood penalty?. Am. J. Sociol. 112 : 1297– 339 [Google Scholar]
• Cox D . 1958 . Planning of Experiments New York: Wiley [Google Scholar]
• Crépon B , Devoto F , Duflo E , Parienté W . 2011 . Impact of microcredit in rural areas of Morocco: evidence from a randomized evaluation. Work. Pap., Dep. Econ., MIT [Google Scholar]
• Cross H , Kenney GM , Mell J , Zimmerman W . 1990 . Employer hiring practices: differential treatment of Hispanic and Anglo job seekers. Tech. rep., Urban Inst., Washington, DC [Google Scholar]
• Deaton A . 2010 . Instruments, randomization, and learning about development. J. Econ. Lit. 48 : 424– 55 [Google Scholar]
• Dehejia R , Pop-Eleches C , Samii C . 2015 . From local to global: external validity in a fertility natural experiment. Work. Pap. 21459, NBER, Cambridge, MA [Google Scholar]
• Doob AN , Gross AE . 1968 . Status as an inhibitor of horn-honking responses. J. Soc. Psychol. 76 : 213– 18 [Google Scholar]
• Druckman JN , Green DP , Kuklinski JH , Lupia A . 2011 . Cambridge Handbook of Experimental Political Science Cambridge, UK: Cambridge Univ. Press [Google Scholar]
• Duflo E , Kremer M , Robinson J . 2008 . How high are rates of return to fertilizer? Evidence from field experiments in Kenya. Am. Econ. Rev. 98 : 482– 88 [Google Scholar]
• Duflo E , Kremer M , Robinson J . 2011 . Nudging farmers to use fertilizer: theory and experimental evidence from Kenya. Am. Econ. Rev. 101 : 2350– 90 [Google Scholar]
• Dunn EW , Aknin LB , Norton MI . 2008 . Spending money on others promotes happiness. Science 319 : 1687– 88 [Google Scholar]
• Dunning T . 2012 . Natural Experiments in the Social Sciences: A Design-Based Approach Cambridge, UK: Cambridge Univ. Press [Google Scholar]
• Dupas P . 2009 . What matters (and what does not) in households’ decision to invest in malaria prevention?. Am. Econ. Rev. 99 : 224– 30 [Google Scholar]
• Dupas P . 2011 . Do teenagers respond to HIV risk information? Evidence from a field experiment in Kenya. Am. Econ. J. Appl. Econ. 3 : 1– 34 [Google Scholar]
• Dupas P . 2014 . Short-run subsidies and long-run adoption of new health products: evidence from a field experiment. Econometrica 82 : 197– 228 [Google Scholar]
• Dupas P , Robinson J . 2011 . Savings constraints and microenterprise development: evidence from a field experiment in Kenya. Work. Pap. 14693, NBER, Cambridge, MA [Google Scholar]
• Emswiller T , Deaux K , Willits JE . 1971 . Similarity, sex, and requests for small favors. J. Appl. Soc. Psychol. 1 : 284– 91 [Google Scholar]
• Enos RD . 2014 . Causal effect of intergroup contact on exclusionary attitudes. PNAS 111 : 3699– 704 [Google Scholar]
• Enos RD , Fowler A , Vavreck L . 2014 . Increasing inequality: the effect of GOTV mobilization on the composition of the electorate. J. Polit. 76 : 273– 88 [Google Scholar]
• Fearon JD , Humphreys M , Weinstein JM . 2009 . Can development aid contribute to social cohesion after civil war? Evidence from a field experiment in post-conflict Liberia. Am. Econ. Rev. 99 : 287– 91 [Google Scholar]
• Fearon JD , Humphreys M , Weinstein JM . 2015 . How does development assistance affect collective action capacity? Results from a field experiment in post-conflict Liberia. Am. J. Political Sci. 109 : 450– 69 [Google Scholar]
• Fershtman C , Gneezy U . 2001 . Discrimination in a segmented society: an experimental approach. Q. J. Econ. 116 : 351– 77 [Google Scholar]
• Fisher RA . 1935 . The Design of Experiments New York: Hafner [Google Scholar]
• Fiszbein A , Schady N . 2009 . Conditional cash transfers: reducing present and future poverty. World Bank Policy Res. Rep., World Bank Washington, DC: [Google Scholar]
• Forbes GB , Gromoll HF . 1971 . The lost letter technique as a measure of social variables: some exploratory findings. Soc. Forces 50 : 113– 15 [Google Scholar]
• Freedman JL , Fraser SC . 1966 . Compliance without pressure: the foot-in-the-door technique. J. Pers. Soc. Psychol. 4 : 195– 202 [Google Scholar]
• Freese J , Peterson D . 2017 . Replication in social science. Annu. Rev. Sociol. 43. In press [Google Scholar]
• Fryer R . 2011 . Financial incentives and student achievement: evidence from randomized trials. Q. J. Econ. 126 : 1755– 98 [Google Scholar]
• Gaddis SM . 2015 . Discrimination in the credential society: an audit study of race and college selectivity in the labor market. Soc. Forces 93 : 1451– 79 [Google Scholar]
• Gaddis SM , Ghoshal R . 2015 . Arab American housing discrimination, ethnic competition, and the contact hypothesis. Ann. Am. Acad. Political Soc. Sci. 660 : 282– 99 [Google Scholar]
• Galster G , Constantine P . 1991 . Discrimination against female-headed households in rental housing: theory and exploratory evidence. Rev. Soc. Econ. 49 : 76– 100 [Google Scholar]
• Gantner L . 2007 . PROGRESA: An integrated approach to poverty alleviation in Mexico. Case Studies in Food Policy for Developing Countries: Policies for Health, Nutrition, Food Consumption, and Poverty P Pinstrup-Andersen, F Cheng, Vol 1 211– 20 Ithaca, NY: Cornell Univ. Press [Google Scholar]
• Garfinkel H . 1967 . Studies in Ethnomethodology Englewood Cliffs, NJ: Prentice-Hall [Google Scholar]
• Gelman A . 2014 . Experimental reasoning in social science. Field Experiments and Their Critics: Essays on the Uses and Abuses of Experimentation in the Social Sciences DL Teele 185– 95 New Haven, CT: Yale Univ. Press [Google Scholar]
• Gerber AS . 2011 . Field experiments in political science. Cambridge Handbook of Experimental Political Science JN Druckman, DP Green, JH Kuklinski, A Lupia 115– 38 Cambridge, UK: Cambridge Univ. Press [Google Scholar]
• Gerber AS , Green DP . 2000 . The effects of canvassing, telephone calls, and direct mail on voter turnout: a field experiment. Am. Political Sci. Rev. 94 : 653– 63 [Google Scholar]
• Gerber AS , Green DP . 2012 . Field Experiments New York: Norton [Google Scholar]
• Gerber AS , Green DP , Larimer CW . 2008 . Social pressure and voter turnout: evidence from a large scale field experiment. Am. Political Sci. Rev. 102 : 33– 48 [Google Scholar]
• Gerber AS , Green DP , Shachar R . 2003 . Voting may be habit-forming: evidence from a randomized field experiment. Am. J. Political Sci. 47 : 540– 50 [Google Scholar]
• Gil-White F . 2004 . Ultimatum game with an ethnicity manipulation: results from Kohvdiin Bulgan Sum, Mongolia. Foundations of Human Sociality: Economic Experiments and Ethnographic Evidence from Fifteen Small-Scale Societies J Henrich, R Boyd, S Bowles, C Camerer, E Fehr, H Gintis, 260– 304 Oxford, UK: Oxford Univ. Press [Google Scholar]
• Gilligan MJ , Pasquale BJ , Samii C . 2014 . Civil war and social cohesion: lab-in-the-field evidence from Nepal. Am. J. Political Sci. 58 : 604– 19 [Google Scholar]
• Giné X , Karlan D . 2014 . Group versus individual liability: short and long term evidence from Philippine-microcredit lending groups. J. Dev. Econ. 107 : 65– 83 [Google Scholar]
• Giné X , Karlan D , Zinman J . 2010 . Put your money where your butt is: a commitment contract for smoking cessation. Am. Econ. J. Appl. Econ. 213– 35 [Google Scholar]
• Gneezy U , List J , Price MK . 2012 . Toward an understanding of why people discriminate: evidence from a series of natural field experiments. Work. Pap. 17855, NBER, Cambridge, MA [Google Scholar]
• Gneezy U , Meier S , Rey-Biel P . 2011 . When and why incentives (don't) work to modify behavior. J. Econ. Perspect. 25 : 191– 210 [Google Scholar]
• Gneezy U , Rey-Biel P . 2014 . On the relative efficiency of performance pay and noncontingent incentives. J. Eur. Econ. Assoc. 12 : 62– 72 [Google Scholar]
• Gneezy U , Rustichini A . 2000 . A fine is a price. J. Legal Stud. 29 : 1– 17 [Google Scholar]
• Goel V . 2014 . Facebook tinkers with users’ emotions in news feed experiment, stirring outcry. New York Times , June 30 B1
• Gosnell HF . 1927 . Getting Out the Vote: An Experiment in the Stimulation of Voting Chicago: Chicago Univ. Press [Google Scholar]
• Green DP , Gerber A . 2008 . Get Out the Vote: How to Increase Voter Turnout Washington, DC: Brookings Inst. Press. 2nd ed. [Google Scholar]
• Green DP , Wong J . 2009 . Tolerance and the contact hypothesis: a field experiment. The Political Psychology of Democratic Citizenship 228– 46 Oxford, UK: Oxford Univ. Press [Google Scholar]
• Greenberg D , Shroder M . 2004 . The Digest of Social Experiments. Washington, DC: Urban Inst. Press [Google Scholar]
• Grose CR . 2014 . Field experimental work on political institutions. Annu. Rev. Political Sci. 17 : 355– 70 [Google Scholar]
• Grossman G , Baldassarri D . 2012 . The impact of elections on cooperation: evidence from a lab in the field experiment in Uganda. Am. J. Political Sci. 56 : 964– 85 [Google Scholar]
• Grossman G , Paler L . 2015 . Using experiments to study political institutions. Handbook of Comparative Political Institutions J Gandhi, R Ruiz-Rufino 84– 97 London: Routledge [Google Scholar]
• Habyarimana J , Humphreys M , Posner DN , Weinstein JM . 2009 . Coethnicity: Diversity and the Dilemmas of Collective Action New York: Russell Sage Found. [Google Scholar]
• Harrison GW . 2013 . Field experiments and methodological intolerance. J. Econ. Methodol. 20 : 103– 17 [Google Scholar]
• Harrison GW , List JA . 2004 . Field experiments. J. Econ. Lit. 42 : 1009– 55 [Google Scholar]
• Hausman JA , Wise DA . 1985 . Social Experimentation Chicago: Chicago Univ. Press [Google Scholar]
• Heckman JJ . 1992 . Randomization and social policy evaluation. Evaluating Welfare and Training Programs CF Manski, I Garfinkel 201– 30 Cambridge, MA: Harvard Univ. Press [Google Scholar]
• Heckman JJ . 1998 . Detecting discrimination. J. Econ. Perspect. 12 : 101– 16 [Google Scholar]
• Heckman JJ , Siegelman P . 1993 . The Urban Institute audit studies: their methods and findings. Clear and Convincing Evidence: Measurement of Discrimination in America M Fix, RJ Struyk 187– 258 Washington, DC: Urban Inst. Press [Google Scholar]
• Henrich J , Boyd R , Bowles S , Camerer C , Fehr E . et al. 2001 . In search of homo economicus: behavioral experiments in 15 small-scale societies. Am. Econ. Rev. 91 : 73– 78 [Google Scholar]
• Henrich J , Ensminger J , McElreath R , Barr A , Barrett C . et al. 2010 . Markets, religion, community size, and the evolution of fairness and punishment. Science 327 : 1480– 84 [Google Scholar]
• Henrich J , McElreath R , Barr A , Ensminger J , Barrett C . et al. 2006 . Costly punishment across human societies. Science 312 : 1767– 70 [Google Scholar]
• Henry PJ . 2008 . College sophomores in the laboratory redux: influences of a narrow data base on social psychology's view of the nature of prejudice. Psychol. Inq. 19 : 49– 71 [Google Scholar]
• Herberich DH , List JA , Price MK . 2011 . How many economists does it take to change a light bulb? A natural field experiment on technology adoption Work. Pap., Univ. Chicago [Google Scholar]
• Heyman J , Ariely D . 2004 . Effort for payment: a tale of two markets. Psychol. Sci. 15 : 787– 93 [Google Scholar]
• Holland J , Silva AS , Mace R . 2012 . Lost letter measure of variation in altruistic behaviour in 20 neighbourhoods. PLOS ONE 7 : e43294 [Google Scholar]
• Houlette MA , Gaertner SL , Johnson KM , Banker BS , Riek BM , Dovidio JF . 2004 . Developing a more inclusive social identity: an elementary school intervention. J. Soc. Issues 60 : 35– 55 [Google Scholar]
• Humphreys M , Sanchez de la Sierra R , van der Windt P . 2013 . Fishing, commitment, and communication: a proposal for comprehensive nonbinding research registration. Polit. Anal. 21 : 1– 20 [Google Scholar]
• Imbens G , Wooldridge J . 2009 . Recent developments in the econometrics of program evaluation. J. Econ. Lit. 47 : 5– 86 [Google Scholar]
• Isen AM , Levin PF . 1972 . Effect of feeling good on helping: cookies and kindness. J. Pers. Soc. Psychol. 21 : 384– 88 [Google Scholar]
• Jackson M , Cox DR . 2013 . The principles of experimental design and their application in sociology. Annu. Rev. Sociol. 39 : 27– 49 [Google Scholar]
• Jensen R , Miller N . 2008 . Giffen behavior and subsistence consumption. Am. Econ. Rev. 98 : 1553– 77 [Google Scholar]
• Kamenica E . 2012 . Behavioral economics and psychology of incentives. Annu. Rev. Econ. 4 : 427– 52 [Google Scholar]
• Karlan D . 2005 . Using experimental economics to measure social capital and predict financial decisions. Am. Econ. Rev. 95 : 1688– 99 [Google Scholar]
• Karlan D , Appel J . 2011 . More Than Good Intentions: Improving the Ways the World's Poor Borrow, Save, Farm, Learn, and Stay Healthy New York: Penguin [Google Scholar]
• Karlan D , Goldberg N . 2011 . Microfinance evaluation strategies: notes on methodology and findings. The Handbook of Microfinance B Armendáriz, M Labie 17– 58 London: World Scientific [Google Scholar]
• Karlan D , McConnell M , Mullainathan S , Zinman J . 2014 . Getting to the top of mind: how reminders increase saving. Manag. Sci. 62 : 3393– 3411 [Google Scholar]
• Karlan D , Osei-Akoto I , Osei R , Udry C . 2010 . Examining underinvestment in agriculture: measuring returns to capital and insurance. Work. Pap., Abdul Latif Jameel Poverty Action Lab. https://www.poverty-action.org/sites/default/files/Panel3-3-Farmers-Returns-Capital.pdf [Google Scholar]
• Karlan D , Zinman J . 2011 . Microcredit in theory and practice: using randomized credit scoring for impact. Science 332 : 1278– 84 [Google Scholar]
• Keizer K , Lindenberg S , Steg L . 2008 . The spreading of disorder. Science 322 : 1681– 85 [Google Scholar]
• Kelly E , Moena P , Oakes J , Fan W , Okechukwu C . et al. 2014 . Changing work and work-family conflict: evidence from the work, family, and health network. Am. Sociol. Rev. 79 : 485– 516 [Google Scholar]
• Kling JR , Liebman JB , Katz LF . 2007 . Experimental analysis of neighborhood effects. Econometrica 75 : 83– 119 [Google Scholar]
• Kotran A . 2015 . Opower and utility partners save over eight terawatt-hours of energy power and utility partners save over eight terawatt-hours of energy. News release, May 21
• Kramer ADI , Guillory JE , Hancock JT . 2014 . Experimental evidence of massive-scale emotional contagion through social networks. PNAS 111 : 8788– 90 [Google Scholar]
• Kremer M . 2003 . Randomized evaluations of educational programs in developing countries: some lessons. Am. Econ. Rev. 93 : 102– 6 [Google Scholar]
• Kremer M , Brannen C , Glennerster R . 2013 . The challenge of education and learning in the developing world. Science 340 : 297– 300 [Google Scholar]
• Kremer M , Leino J , Miguel E , Zwane AP . 2011 . Spring cleaning: rural water impacts, valuation, and property rights institutions. Q. J. Econ. 126 : 145– 205 [Google Scholar]
• Kugelmass H . 2016 . “Sorry, I'm not accepting new patients”: an audit study of access to mental health care. J. Health Soc. Behav. 57 : 168– 83 [Google Scholar]
• Lacetera N , Macis M . 2010 . Do all material incentives for pro-social activities backfire? The response to cash and non-cash incentives for blood donations. J. Econ. Psychol. 31 : 738– 48 [Google Scholar]
• Lacetera N , Macis M , Slonim R . 2013 . Economic rewards to motivate blood donations. Science 340 : 927– 28 [Google Scholar]
• Landry CE , Lange A , List JA , Price MK , Rupp NG . 2010 . Is a donor in hand better than two in the bush? Evidence from a natural field experiment. Am. Econ. Rev. 100 : 958– 83 [Google Scholar]
• Langer EJ , Rodin J . 1976 . The effects of choice and enhanced responsibility for the aged: a field experiment in an institutional setting. J. Pers. Soc. Psychol. 34 : 191– 98 [Google Scholar]
• Lauster N , Easterbrook A . 2011 . No room for new families? A field experiment measuring rental discrimination against same-sex couples and single parents. Soc. Probl. 58 : 389– 409 [Google Scholar]
• Leuven E , Oosterbeek H , van der Klaauw B . 2010 . The effect of financial rewards on students’ achievement: evidence from a randomized experiment. J. Eur. Econ. Assoc. 8 : 1243– 65 [Google Scholar]
• Levine M , Prosser A , Evans D , Reicher S . 2005 . Identity and emergency intervention: how social group membership and inclusiveness of group boundaries shape helping behavior. Pers. Soc. Psychol. Bull. 31 : 443– 53 [Google Scholar]
• Levitt SD , List JA . 2009 . Field experiments in economics: the past, the present, and the future. Eur. Econ. Rev. 53 : 1– 18 [Google Scholar]
• Levitt SD , List JA , Neckerman S , Sadoff S . 2012 . The behavioralist goes to school: leveraging behavioral economics to improve educational performance. Work. Pap. 18165, NBER Cambridge, MA: [Google Scholar]
• List JA . 2007 . Field experiments: a bridge between lab and naturally occurring data. B.E. J. Econ. Anal. Policy 5 : 2 [Google Scholar]
• Lucas JW . 2003 . Theory-testing, generalization, and the problem of external validity. Sociol. Theory 21 : 236– 53 [Google Scholar]
• Ludwig J , Duncan GJ , Gennetian LA , Katz LF , Kessler RC . et al. 2013 . Long-term neighborhood effects on low-income families: evidence from moving to opportunity. Am. Econ. Rev. 103 : 226– 31 [Google Scholar]
• Ludwig J , Liebman JB , Kling JR , Duncan GJ , Katz LF . et al. 2008 . What can we learn about neighborhood effects from the moving to opportunity experiment?. Am. J. Sociol. 114 : 144– 88 [Google Scholar]
• Marwell G , Ames RE . 1979 . Experiments on the provision of public goods: resources, interest, group size, and the free-rider problem. Am. J. Sociol. 84 : 1335– 60 [Google Scholar]
• Massey DS , Lundy G . 2001 . Use of Black English and racial discrimination in urban housing markets: new methods and findings. Urban Aff. Rev. 36 : 452– 69 [Google Scholar]
• McDermott R . 2011 . Internal and external validity. Cambridge Handbook of Experimental Political Science JN Druckman, DP Green, JH Kuklinski, A Lupia, 27– 40 Cambridge, UK: Cambridge Univ. Press [Google Scholar]
• McEwan PJ . 2015 . Improving learning in primary schools of developing countries: a meta-analysis of randomized experiments. Rev. Educ. Res. 85 : 353– 94 [Google Scholar]
• McNutt M . 2015 . Editorial retraction of Lacour & Green. Science 346 : 1366– 69 Science 348 : 1100 [Google Scholar]
• Merton RK . 1945 . Sociological theory. Am. J. Sociol. 50 : 462– 73 [Google Scholar]
• Michelson M , Nickerson DW . 2011 . Voter Mobilization Cambridge, UK: Cambridge Univ. Press [Google Scholar]
• Miguel E , Kremer M . 2004 . Worms: identifying impacts on education and health in the presence of treatment externalities. Econometrica 72 : 159– 217 [Google Scholar]
• Milgram S , Liberty HJ , Toledo R , Wackenhut J . 1986 . Response to intrusion into waiting lines. J. Pers. Soc. Psychol. 51 : 683– 89 [Google Scholar]
• Milgram S , Mann L , Hartner S . 1965 . The lost letter technique: a tool of social research. Public Opin. Q. 29 : 437– 38 [Google Scholar]
• Milkman KL , Akinola M , Chugh D . 2015 . What happens before? A field experiment exploring how pay and representation differentially shape bias on the pathway into organizations. J. Appl. Psychol. 100 : 1678– 712 [Google Scholar]
• Milkman KL , Beshears J , Choi JJ , Laibson D , Madrian BC . 2011 . Using implementation intentions prompts to enhance influenza vaccination rates. PNAS 108 : 10415– 20 [Google Scholar]
• Morgan S , Winship C . 2007 . Counterfactuals and Causal Inference Cambridge, UK: Cambridge Univ. Press [Google Scholar]
• Morton R , Williams K . 2010 . Experimental Political Science and the Study of Causality Cambridge, UK: Cambridge Univ. Press [Google Scholar]
• Moss-Racusin CA , Dovidio JF , Brescoll V , Graham MJ , Handelsman J . 2012 . Science faculty's subtle gender biases favor male students. PNAS 109 : 16474– 79 [Google Scholar]
• Munnell AH . 1986 . Lessons from the Income Maintenance Experiments Boston: Fed. Res. Bank of Boston [Google Scholar]
• Mutz DC . 2011 . Population-Based Survey Experiments Princeton, NJ: Princeton Univ. Press [Google Scholar]
• Nagda BRA , Tropp LR , Paluck EL . 2006 . Looking back as we look ahead: integrating research, theory, and practice on intergroup relations. J. Soc. Issues 62 : 439– 51 [Google Scholar]
• Neumark D , Bank RJ , Nort KDV . 1996 . Sex discrimination in restaurant hiring: an audit study. Q. J. Econ. 111 : 915– 41 [Google Scholar]
• Nickerson DW . 2008 . Is voting contagious? Evidence from two field experiments. Am. Political Sci. Rev. 102 : 49– 57 [Google Scholar]
• Nolan JM , Kenefick J , Schultz PW . 2011 . Normative messages promoting energy conservation will be underestimated by experts unless you show them the data. Soc. Influence 6 : 169– 80 [Google Scholar]
• Nolan JM , Schultz PW , Cialdini RB , Goldstein NJ , Griskevicius V . 2008 . Normative social influence is underdetected. Pers. Soc. Psychol. Bull. 34 : 913– 23 [Google Scholar]
• Nosek B , Aarts A , Anderson J , Anderson C , Attridge P . et al. 2015a . Estimating the reproducibility of psychological science. Science 349 : 943– 51 [Google Scholar]
• Nosek B , Alter G , Banks G , Borsboom D , Bowman S . et al. 2015b . Promoting an open research culture. Science 348 : 1422– 25 [Google Scholar]
• Olken B . 2007 . Monitoring corruption: evidence from a field experiment in Indonesia. J. Political Econ. 115 : 200– 49 [Google Scholar]
• Olken B . 2010 . Direct democracy and local public goods: evidence from a field experiment in Indonesia. Am. Political Sci. Rev. 104 : 243– 67 [Google Scholar]
• Pager D . 2003 . The mark of a criminal record. Am. J. Sociol. 108 : 937– 75 [Google Scholar]
• Pager D . 2007 . The use of field experiments for studies of employment discrimination: contributions, critiques, and directions for the future. Ann. Am. Acad. Political Soc. Sci. 609 : 104– 33 [Google Scholar]
• Pager D , Quillian L . 2005 . Walking the talk: what employers say versus what they do. Am. Sociol. Rev. 70 : 355– 80 [Google Scholar]
• Pager D , Western B , Bonikowski B . 2009 . Discrimination in a low-wage labor market: a field experiment. Am. Sociol. Rev. 74 : 777– 99 [Google Scholar]
• Paluck EL . 2009 . Reducing intergroup prejudice and conflict using the media: a field experiment in Rwanda. Interpers. Relat. Group Process. 96 : 574– 87 [Google Scholar]
• Paluck EL , Cialdini RB . 2014 . Field research methods. Handbook of Research Methods in Social and Personality Psychology HT Reis, CM Judd 81– 97 New York: Cambridge Univ. Press, 2nd ed.. [Google Scholar]
• Paluck EL , Green DP . 2009 . Prejudice reduction: what works? A review and assessment of research and practice. Annu. Rev. Psychol. 60 : 339– 67 [Google Scholar]
• Paluck EL , Shepherd H . 2012 . The salience of social referents: a field experiment on collective norms and harassment behavior in a school social network. J. Pers. Soc. Psychol. 103 : 899– 915 [Google Scholar]
• Paluck EL , Shepherd H , Aronow PM . 2016 . Changing climates of conflict: a social network driven experiment in 56 schools. PNAS 113 : 566– 71 [Google Scholar]
• Pedulla DS . 2016 . Penalized or protected? Gender and the consequences of non-standard and mismatched employment histories. Am. Sociol. Rev. 81 : 262– 89 [Google Scholar]
• Pettigrew TF . 1998 . Intergroup contact theory. Annu. Rev. Psychol. 49 : 65– 85 [Google Scholar]
• Riach PA , Rich J . 2002 . Field experiments of discrimination in the market place. Econ. J. 112 : 480– 518 [Google Scholar]
• Rodríguez-Planas N . 2012 . Longer-term impacts of mentoring, educational services, and learning incentives: evidence from a randomized trial in the United States. Am. Econ. J. Appl. Econ. 4 : 121– 39 [Google Scholar]
• Rondeau D , List JA . 2008 . Matching and challenge gifts to charity: evidence from laboratory and natural field experiments. Exp. Econ. 11 : 253– 67 [Google Scholar]
• Ross SL , Turner MA . 2005 . Housing discrimination in metropolitan America: explaining changes between 1989 and 2000. Soc. Probl. 52 : 152– 80 [Google Scholar]
• Rossi PH , Berk RA , Lenihan KJ . 1980 . Money, Work, and Crime: Experimental Evidence New York: Academic Press [Google Scholar]
• Rossi PH , Berk RA , Lenihan KJ . 1982 . Saying it wrong with figures: a comment on Zeisel. Am. J. Sociol. 88 : 390– 93 [Google Scholar]
• Rossi PH , Lyall KC . 1978 . An overview evaluation of the NIT experiment. Eval. Stud. Rev. 3 : 412– 28 [Google Scholar]
• Sabin N . 2015 . Modern microfinance: a field in flux. Social Finance Nicholls A, Paton R, Emerson J Oxford, UK: Oxford Univ. Press [Google Scholar]
• Salganik MJ , Dodds PS , Watts DJ . 2006 . Experimental study of inequality and unpredictability in an artificial cultural market. Science 311 : 854– 56 [Google Scholar]
• Sampson RJ . 2008 . Moving to inequality: neighborhood effects and experiments meet social structure. Am. J. Sociol. 114 : 189– 231 [Google Scholar]
• Sampson RJ . 2012 . Great American City: Chicago and the Enduring Neighborhood Effect Chicago, IL: Chicago Univ. Press [Google Scholar]
• Schuler SR , Hashemi SM , Badal SH . 1998 . Men's violence against women in rural Bangladesh: undermined or exacerbated by microcredit programmes?. Dev. Pract. 8 : 148– 57 [Google Scholar]
• Schultz P . 2004 . School subsidies for the poor: evaluating the Mexican Progresa poverty program. J. Dev. Econ. 74 : 199– 250 [Google Scholar]
• Shadish WR , Cook TD . 2009 . The renaissance of field experimentation in evaluating interventions. Annu. Rev. Psychol. 607– 29 [Google Scholar]
• Shadish WR , Cook TD , Campbell DT . 2002 . Experimental and Quasi-experimental Designs for Generalized Causal Inference. New York: Houghton, Mifflin and Company [Google Scholar]
• Simpson BT , McGrimmon T , Irwin K . 2007 . Are blacks really less trusting than whites? Revisiting the race and trust question. Soc. Forces 86 : 525– 52 [Google Scholar]
• Sniderman PM , Grob DB . 1996 . Innovations in experimental design in attitude surveys. Annu. Rev. Sociol. 22 : 377– 99 [Google Scholar]
• Steinpreis RE , Anders KA , Ritzke D . 1999 . The impact of gender on the review of the curricula vitae of job applicants and tenure candidates: a national empirical study. Sex Roles 41 : 509– 28 [Google Scholar]
• Stutzer A , Goette L , Zehnder M . 2011 . Active decisions and prosocial behaviour: a field experiment on blood donations. Econ. J. 121 : 476– 93 [Google Scholar]
• Teele DL . 2014 . Reflections on the ethics of field experiments. Field Experiments and Their Critics: Essays on the Uses and Abuses of Experimentation in the Social Sciences DL Teele 115– 40 New Haven, CT: Yale Univ. Press [Google Scholar]
• Thornton RL . 2008 . The demand for, and impact of, learning HIV status. Am. Econ. Rev. 98 : 1829– 63 [Google Scholar]
• Tilcsik A . 2011 . Pride and prejudice: employment discrimination against openly gay men in the United States. Am. J. Sociol. 117 : 586– 626 [Google Scholar]
• Travers J , Milgram S . 1969 . An experimental study of the small world problem. Sociometry 32 : 425– 43 [Google Scholar]
• Turner MA , Bednarz BA , Herbig C , Lee SJ . 2003 . Discrimination in metropolitan housing markets phase 2: Asians and Pacific Islanders Tech. rep., Urban Inst., Washington, DC [Google Scholar]
• Turner MA , Fix M , Struyk RJ . 1991 . Opportunities Denied, Opportunities Diminished: Racial Discrimination in Hiring Washington, DC: Urban Inst. Press [Google Scholar]
• Turner MA , Ross SL , Galster GC , Yinger J . 2002 . Discrimination in metropolitan housing markets: national results from phase 1 of the Housing Discrimination Study (HDS) Tech. rep., Urban Inst Washington, DC: [Google Scholar]
• Van Bavel JJ , Mende-Siedlecki P , Brady WJ , Reinero DA . 2016 . Contextual sensitivity in scientific reproducibility. PNAS 113 : 6454– 59 [Google Scholar]
• Van de Rijt A , Kang SM , Restivo M , Patil A . 2014 . Field experiments of success-breeds-success dynamics. PNAS 111 : 6934– 39 [Google Scholar]
• Van Der Merwe WG , Burns J . 2008 . What's in a name? Racial identity and altruism in post-apartheid South Africa. South Afr. J. Econ. 76 : 266– 75 [Google Scholar]
• Vermeersch C , Kremer M . 2005 . School Meals, Educational Achievement, and School Competition: Evidence from a Randomized Evaluation. New York: World Bank [Google Scholar]
• Volpp KG , Troxel AB , Pauly MV , Glick HA , Puig A . et al. 2009 . A randomized, controlled trial of financial incentives for smoking cessation. N. Engl. J. Med. 360 : 699– 709 [Google Scholar]
• Whitt S , Wilson RK . 2007 . The dictator game, fairness and ethnicity in postwar Bosnia. Am. J. Political Sci. 51 : 655– 68 [Google Scholar]
• Wienk RE , Reid CE , Simonson JC , Eggers FJ . 1979 . Measuring racial discrimination in American housing markets: the housing market practices survey. Tech. Rep. HUD-PDR-444(2), Dep. Hous. Urban Dev Washington, DC: [Google Scholar]
• Williams WM , Ceci SJ . 2015 . National hiring experiments reveal 2:1 faculty preference for women on STEM tenure track. PNAS 112 : 5360– 65 [Google Scholar]
• Yamagishi T . 2011 . Trust: The Evolutionary Game of Mind and Society New York: Springer [Google Scholar]
• Yamagishi T , Cook KS , Watabe M . 1998 . Uncertainty, trust, and commitment formation in the United States and Japan. Am. J. Sociol. 104 : 165– 94 [Google Scholar]
• Zeisel H . 1982 . Disagreement over the evaluation of a controlled experiment. Am. J. Sociol. 88 : 378– 89 [Google Scholar]

Data & Media loading...

• Article Type: Review Article

Most Read This Month

Most cited most cited rss feed, birds of a feather: homophily in social networks, social capital: its origins and applications in modern sociology, conceptualizing stigma, framing processes and social movements: an overview and assessment, organizational learning, the study of boundaries in the social sciences, assessing “neighborhood effects”: social processes and new directions in research, social exchange theory, culture and cognition, focus groups.

Field Experiments

• Reference work entry
• First Online: 01 January 2018
• pp 4569–4574
• Cite this reference work entry

• John A. List 1 &
• David Reiley 1  

33 Accesses

Field experiments have grown significantly in prominence since the 1990s. In this article, we provide a summary of the major types of field experiments, explore their uses, and describe a few examples. We show how field experiments can be used for both positive and normative purposes within economics. We also discuss more generally why data collection is useful in science, and more narrowly discuss the question of generalizability. In this regard, we envision field experiments playing a classic role in helping investigators learn about the behavioural principles that are shared across different domains.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save.

• Get 10 units per month
• Download Article/Chapter or eBook
• 1 Unit = 1 Article or 1 Chapter
• Cancel anytime
• Available as PDF
• Read on any device
• Instant download
• Own it forever
• Available as EPUB and PDF
• Durable hardcover edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Exploiting Data from Field Experiments

Online field experiments: a selective survey of methods

Bibliography.

Anderson, E.T., and D. Simester. 2003. Effects of $9 price endings on retail sales: Evidence from field experiments. Quantitative Marketing and Economics 1: 93–110.

Article   Google Scholar  

Bohm, P. 1972. Estimating the demand for public goods: An experiment. European Economic Review 3: 111–130.

Engelmann, D. and V. Grimm. 2003. Bidding behavior in multi-unit auctions – An experimental investigation and some theoretical insights. Working paper. Prague: Centre for Economic Research and Graduate Education, Economic Institute.

Google Scholar  

Harrison, G.W., and J.A. List. 2004. Field experiments. Journal of Economic Literature 42: 1009–1055.

Karlan, D. and J.A. List. 2007. Does price matter in charitable giving? Evidence from a large-scale natural field experiment. American Economic Review , forthcoming.

Levitt, S.D., and J.A. List. 2006. What do laboratory experiments measuring social preferences tell us about the real world? Journal of Economic Perspectives 21(2): 153–174.

List, J.A. 2006. Field experiments: A bridge between lab and naturally occurring data. Advances in Economic Analysis & Policy 6(2), Article 8. Abstract online. http://www.bepress.com/beieap/advances/vol6/iss2/art8 . Accessed 26 May 2007.

List, J.A., and D. Lucking-Reiley. 2000. Demand reduction in a multi-unit auction: Evidence from a sportscard field experiment. American Economic Review 90: 961–972.

Lucking-Reiley, D. 1999. Using field experiments to test equivalence between auction formats: Magic on the Internet. American Economic Review 89: 1063–1080.

Orne, M.T. 1962. On the social psychological experiment: With particular reference to demand characteristics and their implications. American Psychologist 17: 776–783.

Porter, D. and R. Vragov. 2003. An experimental examination of demand reduction in multi-unit versions of the uniform-price, Vickrey, and English auctions. Working paper. Interdisciplinary Center for Economic Science, George Mason University.

Vickrey, W. 1961. Counterspeculation, auctions, and competitive sealed tenders. Journal of Finance 16: 8–37.

Download references

Author information

Authors and affiliations.

http://link.springer.com/referencework/10.1057/978-1-349-95121-5

John A. List & David Reiley

You can also search for this author in PubMed   Google Scholar

Editor information

Copyright information.

© 2018 Macmillan Publishers Ltd.

About this entry

Cite this entry.

List, J.A., Reiley, D. (2018). Field Experiments. In: The New Palgrave Dictionary of Economics. Palgrave Macmillan, London. https://doi.org/10.1057/978-1-349-95189-5_2000

Download citation

DOI : https://doi.org/10.1057/978-1-349-95189-5_2000

Published : 15 February 2018

Publisher Name : Palgrave Macmillan, London

Print ISBN : 978-1-349-95188-8

Online ISBN : 978-1-349-95189-5

eBook Packages : Economics and Finance Reference Module Humanities and Social Sciences Reference Module Business, Economics and Social Sciences

Share this entry

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

The PMC website is updating on October 15, 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Springer Nature - PMC COVID-19 Collection

Lab-in-the-field experiments: perspectives from research on gender

Lata gangadharan.

1 Monash University, Melbourne, Australia

2 Indian Institute of Management, Ahmedabad, India

Pushkar Maitra

3 University of Gothenburg, Gothenburg, Sweden

This paper highlights the contributions made by lab-in-the-field experiments, which are also known as artefactual, framed and extra-lab experiments. We present a curated sample of lab-in-the-field experiments and discuss how they can be conducted on their own or combined with conventional laboratory experiments, natural experiments, randomised control trials and surveys to provide unique insights into the behaviour of a diverse population. Using our recent research on gender and leadership, we demonstrate how lab-in-the-field experiments have offered new perspectives about gender differences in decision-making. Finally, we outline the ethical and implementational challenges researchers may face while conducting these experiments and share some of the strategies we employed to address them.

Introduction

A defining feature of economics is that it investigates how the decisions of agents in society might change in the presence of competing incentives and diverse institutions. These decisions are often modelled using economic theory, yet assessing the empirical robustness of these theories is important for understanding how agents respond to economic and social stimuli in practice. Modern economic theory now recognizes that, in interactions where information asymmetries and expectations matter, agents act strategically subject to a range of factors, including cognitive limitations, preferences that contain elements of fairness, altruism and reciprocity, and social norms. Where these constraints were once lumped in the black box of “unobservables” (often categorized as “omitted variables” in econometric analyses), there is now a growing focus on improving our understanding of them, and how they affect agents’ behaviour (especially in ways that differ from what a traditional rational agent model would predict). While econometric and statistical tools allow us to test theoretical intuition using observational data, such naturally occurring data is often unable to accurately measure behavioural variables, and may not satisfy the assumptions required to provide clean causal inferences. Economic experiments are valuable tools to address these concerns.

Experiments can both help weed out incorrect theories and generate new causal insights that further economic theory as well as improve policy recommendations. In addition, where research questions are motivated by empirical regularities, such as the impact of individual (e.g., gender) and behavioural (e.g., confidence) characteristics on labour market outcomes, well-established evidence from experimental data can enable researchers to identify these regularities and further refine models of economic behaviour.

While early experiments in economics focused on testing specific aspects of economic theory (e.g., functioning of markets, predictions of game-theoretic models of behaviour, and individual choices), experiments are now used in a wide range of areas within economics including public economics, environmental economics, development economics and macroeconomics. The field also provides opportunities for interdisciplinary research due to its connections with psychology, political science, philosophy and sociology.

Researchers have designed both laboratory and field experiments in economics, with each having distinct advantages and limitations. In this paper, we explore the role played by field experiments in economics and focus on a particular kind of field experiment, referred to as a “lab-in-the-field” experiment. Our aim is not to offer a survey of the literature but to highlight some specific contributions of this method and provide select examples of the applications lab-in-the-field experiments can have in economics. Accordingly, we document topics that have benefited from the use of this method and discuss some of our recent research utilising lab-in-the-field experiments to understand gender differences in decision making and leadership. In the last section, we highlight the ethical and implementational challenges we encountered while planning and conducting these experiments, and share the strategies we employed to overcome these difficulties.

What are field experiments?

Field experiments combine naturally occurring field data with aspects of controlled laboratory experiments, harnessing the benefits of randomisation in an environment that captures important features of the real world. This is particularly valuable in the social sciences since research subjects tend to have complex, heterogeneous behaviours, and sampling populations from different domains can permit stronger inferences (List & Reiley, 2008 ).

Economists have taken several approaches to classifying field experiments among other methodologies. Harrison and List ( 2004 ) identify six main distinguishing criteria: (1) the nature of the subject pool; (2) the nature of the information that the subjects bring to the task; (3) the nature of the commodity; (4) the nature of the stakes; (5) the nature of the task or trading rules applied; and (6) the nature of the environment that participants operate in. Using these criteria, they categorize experiments into four groups:

• Conventional lab experiments: those that use a standard subject pool of students;
• Artefactual field experiments: conventional lab experiments that use a “non-standard” subject pool
• Framed field experiments: artefactual field experiments with “field context in either the commodity task, or information set that the subjects can use” (Harrison & List, 2004 )
• Natural field experiment: framed field experiments where the environment is one in which subjects naturally undertake the tasks being studied (such that participants do not know that they are in an experiment).

Charness et al. ( 2013 ) propose an alternative classification that comprise lab, field and extra-lab experiments. Under their approach, extra-lab experiments are similar to conventional lab experiments except for the venue and participant pool (which could include school students, online communities or villagers). Extra-lab experiments have similar characteristics to artefactual and framed field experiments in Harrison and List ( 2004 ). Samek ( 2019 ) provides a comprehensive summary of the advantages and disadvantages of field experiments. In recent years, researchers have also used the term lab-in-the-field experiments to refer to artefactual, framed and extra-lab experiments. This is the term we use in this paper.

Figures  1 and ​ and2 2 present two alternative means of classifying experiments. Figure  1 uses the control that researchers have on the experimental environment, and how aware participants are about being in a research project, as the dimensions for classification. This figure refers to both natural field experiments and natural experiments. Though they share some features, in a natural field experiment, the researcher can often control the randomization, whereas in the natural experiment approach the researcher aims to find sources of variation in existing data that are “as good as randomly assigned” List and Rasul ( 2011 ). Randomised controlled trials (RCTs), which are also very popular in economics, share some characteristics with natural field experiments, however, a point of contrast is that RCT subjects are often aware that they are participating in a study as researchers elicit responses from them at various points throughout the research project. 1 Figure  2 uses to control and ability to repeat the experiment as markers for classification. Figure  2 also includes online experiments, which have seen significant growth in recent years. Online experiments can be thought of as a form of lab-in-the-field experiment since they usually use non-student subjects as participants.

Classification of Experiments by Control and Awareness

Classification of Experiments by Control and Ability to Repeat

While these two-dimensional figures significantly simplify the nuances and characteristics of experimental research, they provide insight into the multi-dimensional choices and tradeoffs researchers face when choosing between methods. Which method is more preferable in a particular case depends primarily on the research question being investigated and the resources available to the researcher.

Overall, field experiments differ from conventional lab experiments along with a number of dimensions, including the subject pool, information that participants have, commodity, tasks, stakes and environment. Field experiments use “non-standard” subjects to the extent that they do not usually involve university students, the typical subject pool for conventional lab experiments. Instead, participants are drawn from the specific target population(s) in the economy to draw inferences from the unique characteristics, information or experiences they bring to the experiment and the decision-making process. In addition, field experiments often elicit choices over actual commodities (rather than abstract commodities, which is typically the case in lab experiments) to better account for whether the commodity itself affects behaviour (e.g. decisions about a real public good such as a road, as in, Beath et al. 2018 ).

Another point of contrast between lab and field experiments is that the latter provide context to suggest strategies and heuristics for undertaking the task at hand. Rather than viewing these as uncontrolled effects, the environmental cues and/or field experiences become central to determining behaviour and the nature of the task that the participants are being asked to perform. Accordingly, although the field context can reduce experimenters’ control, it also has the potential to increase the relevance and saliency of the task, allowing researchers to elicit more accurate responses from participants (depending on the research question). Both Harrison and List ( 2004 ) and Charness et al. ( 2013 ) recommend that field approaches be considered methodologically complementary to lab experiments.

By attempting to simulate “real life” as closely as possible, field experiments can also enhance the external validity, or generalizability, of experimental findings. In this way, field experiments can offer a distinctive and new source of empirical evidence, which can then be compared, contrasted, reconciled and eventually intertwined with evidence from non-experimental and lab methods. 2

Lab-in-the-field experiments

The focus of this paper is on a specific type of field experiment referred to as a lab-in-the-field experiment, also known as an artefactual, framed or extra-laboratory experiment. Viceisza ( 2015 ) provides a survey of the literature on lab-in-the-field experiments.

In economics, lab-in-the-field experiments have been used: (1) to examine the decisions of a broader range of subjects, extending our understanding of human behaviour beyond a few select communities in the western world (Heinrich et al., 2010 ); (2) to compare decisions across subject pools; and (3) in combination with other empirical approaches, such as conventional lab experiments, natural experiments (comprising both natural shocks and policies), randomized control trials and observational data. In this section, we provide examples of each case to demonstrate the contributions made by this experimental approach.

Lab-in-the-field experiments to examine subject pool differences : Since lab-in-the-field experiments employ participants from specific contexts, they can help examine the behaviour of unique populations and determine whether this behaviour is consistent across subject pools. Reflecting the breadth of possible subjects, researchers have conducted experiments with public servants (Alatas et al., 2009 ), nurses (Cadsby & Maynes, 1998 ; Barr et al., 2009 ; Hanna & Wang, 2017 ); CEOs (Fehr & List, 2010 ; List & Mason, 2011 ), managers in Chinese state-owned enterprises (Cooper et al., 1999 ), married couples (Dasgupta & Mani, 2015 ; Iversen et al., 2011 ; Masekesa & Munro, 2020 ), prisoners (Cameron et al., 2019 ; Guo et al., 2020 ) and politicians (Banerjee et al., 2020 ; Chaudhuri et al., 2020 , 2021 ). 3

The study by Alatas et al. ( 2009 ) demonstrates the usefulness of using non-standard subjects. It finds that while both Indonesian public servant participants and Indonesian student subjects are corrupt, public servants are significantly less likely to engage in corruption than student subjects. Using public servants as participants allowed the researchers to tease out the mechanisms underlying their results, revealing that experience (rather than selection into public service) is the main driver of the behaviour. Similarly, Hanna and Wang ( 2017 ) used a sample of students and government workers (nurses) to find that dishonest individuals within both groups prefer to enter government service. More recently, Chaudhuri et al . ( 2020 ) and Chaudhuri et al. ( 2021 ) compare the behaviour of politicians and ordinary citizens using survey and experimental data covering village councils in rural India. They find that inexperienced (and first-time elected) village council politicians are less dishonest and more pro-social than ordinary citizens. However, this idealism appears to wear off over time.

Lab-in-the-field experiments as a complement to traditional laboratory experiments : The lab-in-the-field experiment can be used ex post to test for external validity. For example, Blackburn et al. ( 1994 ) estimated a statistical model of subject response using two different convenience samples: college students, and field subjects drawn from a broad range of churches in the same urban area. The church sample exhibited a much wider variability in socio-demographic characteristics, with ages ranging from 21 to 79 years compared to 19–27 years for the student subject pool. They found that predicting the behaviour of students based on the church-estimated behavioural model was extremely accurate, but predicting church behaviour from the student-estimated behavioural model led to wide forecast variances. It follows that, by offering access to a broader range of research subjects, lab-in-the-field experiments can improve predictions of the behaviour of the target population.

Lab-in-the-field experiments to study responses to natural shocks : Natural occurrences and disasters have often been used by researchers to understand decision-making as the disaster can be considered an exogenous shock. Lab-in-the-field experiments have been conducted in areas which were exposed (more/less) to a disaster to exploit this plausibly exogenous variation. For example, evidence from developed countries is provided by Eckel et al. ( 2009 ), Page et al. ( 2014 ) and Hanaoka et al. ( 2018 ), while evidence from developing countries is supplied in Cameron and Shah ( 2015 ), Cassar et al. ( 2017 ), Brown et al. ( 2018 ) and Islam et al . ( 2020 ). Maitra and Neelim ( 2021 ) summarize some of the recent studies on this topic. The COVID-19 pandemic is an important example of a natural shock the world is currently experiencing. Campos-Mercade et al. ( 2021 ) examine individual responses to this pandemic and find that pro-sociality predicts health behaviour during the pandemic.

In addition, lab-in-the-field experiments have been used to examine the long-term effects of exposure to violent conflicts, wars and resource scarcity. For example, Prediger et al. ( 2014 ), Cecchi and Duchoslav ( 2018 ) and Gangadharan et al . ( 2020 ) find that resource scarcity and exposure to wars (including growing up during wars) can have long term behavioural effects, especially on antisocial behaviour.

Lab-in-the-field experiments to better understand the effect of policies and programs : Researchers have also employed lab-in-the-field experiments in settings where important new policies with overarching impacts have been initiated by governments. Economists have used the exogenous shift in policies to provide useful insights into how a policy change influences savings, consumption patterns, education and labor market outcomes (mainly using observational data). However, measuring and examining behavioural patterns using such data is difficult. In this situation, lab-in-the-field experiments can be leveraged to explore these patterns in a robust way. For example, Cameron et al. ( 2013 ) use the one-child policy in China as a backdrop to examine the social, risk and competitive preferences of those who were single children as a consequence of this policy, while Gangadharan et al. ( 2016 ), and Gangadharan et al. ( 2019 ) (discussed in more detail in Sect.  4 ) use the randomised affirmative action policy in India to investigate gender differences in leadership in a public goods experiment.

One concern when investigating the effect of policy changes that cover large populations, and therefore use large data sets, is that data collection is not targeted to address specific questions. Even where conducting a sophisticated data collection exercise specifically targeted to the program is possible ex post , pre-program data is generally difficult to obtain. Policymakers have found lab-in-the-field experiments to be particularly useful in such cases. For example, Ludwig et al. ( 2011 ) emphasize the importance of uncovering the mechanism through which treatment effects occur in complex policy environments; an insight that in many cases can be derived from a relatively simple set of experimental treatments, “especially when the extra-lab (lab in the field) result is benchmarked against its lab precursor” (Charness et al., 2013 ). By conducting lab-in-the-field experiments directly on a target population or at the location of a potential policy intervention, policy makers can examine its effect on a small scale before fully implementing a project with potentially large consequences. Plott ( 1982 ) and Smith ( 1994 ) mention this approach as being similar to a testbed, which allows for rigorous, transparent and replicable testing of new scientific methods and ideas. Such testbeds have been widely used to develop and fine-tune policies (e.g. in pollution markets and spectrum auctions) in many parts of the world.

Lab-in-the-field experiments combined with Randomised Controlled Trials : RCTs are explicitly designed with a research purpose in mind, with participants randomly assigned to treatment and control arms. RCTs measure outcomes that can accurately be captured using survey data. For instance, an RCT that varies access to cash transfers will generally measure changes in income, consumption and poverty through baseline and end-line surveys. However, as discussed by Falk and Heckman ( 2009 ) and Barrett and Carter ( 2010 ) among others, many RCTs fail to measure and illuminate the mechanisms behind any variation in outcomes, especially behavioural mechanisms. In contrast, lab-in-the-field experiments are specifically designed to understand and measure behavioural mechanisms. 4 For example, Maitra and Mani ( 2017 ) use a lab-in-the-field experiment to examine the role played by risk attitudes and preferences for competitiveness in understanding the causal (treatment) effects obtained from a labor market training program.

Lab-in-the-field experiments are particularly useful where behavioural mechanisms are relevant as they are generally less costly than RCTs and allow for multiple observations of the same individual under varying but controlled conditions. For instance, subjects can participate in a public goods task across multiple periods, with a different partner assigned each period. In addition, lab-in-the-field experiments can test very specific theoretical hypotheses, which is difficult with RCTs because controlling variation in the decision environment in a naturally occurring setting is often not feasible.

As such, lab-in-the-field experiments are useful complements to a range of field experiments, from those arising to examine the impact of disasters or the evaluation of government policies and programs (referred to as natural experiments in Figs.  1 and ​ and2) 2 ) to RCTs examining the effectiveness of the treatment imposed by researchers or policymakers. Despite a somewhat artificial setting, they offer greater control over the environment which allows for a better understanding of the causal mechanisms. This suggests conducting both studies in tandem will improve interpretation as well as the replicability of findings.

Lab-in-the-field experiments as a complement to surveys : In cases where responses to survey questions are prone to social desirability bias, lab-in-the-field experiments can provide a more accurate instrument compared with field surveys. For topics such as discrimination, corruption, dishonesty, fairness and redistribution, direct questions can lead to biased results because the respondent may be more inclined to provide an answer that is socially acceptable rather than reflective of their true attitudes or preferences. For example, on a survey investigating attitudes towards daughters working outside the home, parents may report approving of such behaviour only because this response is socially acceptable. They may also do this to project a favorable image to the surveyor or themselves, or to avoid receiving negative evaluations. While a researcher could test and reduce social desirability bias using methods such as list experiments (Blair & Imai, 2012 ), bounding demand effects (de Quidt et al., 2018 )) or testing sensitivity to report socially desirable responses (Dhar et al., 2020 ), an alternative is to conduct a lab-in-the-field experiment. In experiments, participants may not know the true aim of the design, which reduces the probability that subjects will try to behave in a socially desirable manner.

The fact that, in lab-in-the-field experiments, financial incentives are associated with the choices made by participants within the experiment means that participants are more likely to reveal their true preferences. This is important because monetary payments help link choices to behaviour outside the experimental setting (as choices in the real world often involve payoff consequences) and encourage subjects to take their decisions seriously (Cardenas, 2000 ; Cardenas & Ostrom, 2004 ; Smith, 1989 ); behaviour in the laboratory becomes reliable and “real” when subjects take decisions with meaningful economic consequences because they perceive their own behaviour as relevant and experience real emotions. 5 Researchers have experimentally tested the effect of incentives in experimental economics and have shown that in many (but not all) tasks, subjects exert more effort if monetary earnings are tied to performance. For example, Camerer and Hogarth ( 1999 ), Gneezy and Rustichini ( 2000 ) and Erkal et al. ( 2018 ) provide lab and field evidence on the impact of monetary incentives. Relatedly, researchers have shown that stake size matters. For instance, using a lab-in-the-field experiment, Leibbrandt et al. ( 2018 ) show that participants are more likely to lie when the stakes are very large. Related to this, lab-in-the-field experiments enable researchers to understand and measure behavioural characteristics that have traditionally been considered unobservable in surveys (e.g. preferences, beliefs and norms) as either the dependent or independent variable “at a refined level that is unlikely to be feasible by other empirical methods” (Viceisza, 2015 ). In the subsection below, we outline some of these characteristics.

For the reasons mentioned above, researchers are increasing combining survey and lab-in-the-field methods. For instance, Bartling et al. ( 2009 ) report results from several economic experiments embedded in a household survey study within the German Socio-Economic Panel, a large representative survey of private households in Germany. Sarsons et al. ( 2021 ) similarly use administrative data on publications in economics journals to determine gender differences in evaluations and complement this with an online lab in the field experiments to examine the mechanisms underlying their results. Bhalotra et al . (2021) also combine surveys and lab in the field experiments to understand responses to the religion of a group leader.

Lab-in-the-field experiments: some commonly used games/tasks

Lab-in-the-field experiments have many applications, and most tasks conducted in a conventional lab have the potential to be implemented in the field with some modification. In this section, we provide a brief outline of the most common lab-in-the-field tasks. These include individual choice experiments (such as those conducted to elicit risk and time preferences), experiments to elicit other-regarding preferences and experiments to elicit beliefs about behaviour and social norms.

Risk and Time Preferences : A large body of research shows that individual risk preferences play an important role in decision-making across many domains. For example, risk preferences have significant effects on occupational choices, schooling decisions, technology adoption, financial decisions and the choice to be enrolled in skills training programs (see Castillo et al., 2010 ; Dercon & Christiaensen, 2011 ; Belzil and Leonardi, 2009 ; Liu, 2013 ; Dasgupta et al., 2015 ). Maitra and Neelim ( 2021 ) provide a brief survey of this literature. In recent work, the risk preferences of individuals have also been shown to be relatively stable in developed countries but highly unstable in developing countries (Cardak et al., 2021 ). Risk preferences are often elicited with the use of a lottery. Depending on the elicitation method, subjects can be classified as risk averse, risk neutral or risk takers. Laboratory experiments on risk (such as Eckel & Grossman, 2008 ; Gneezy & Potters, 1997 ; Holt & Laury, 2002 ) were inspired by the early studies of Binswanger ( 1980 ), who conducted risk elicitation experiments with farmers in India.

Time preference, which describes the amount someone values the future relative to the present, is another fundamental factor for explaining decisions. It has been shown that impatient individuals may be reluctant to save or to invest in their future (see Golsteyn et al., 2014 ; Newell & Siikamäki, 2015 ), while more patient women may have a greater incentive to educate themselves and their children, since education may be understood as a long-term investment in the future (Duflo, 2012 ). Individuals may also exhibit naiveté, such that they underestimate the degree of future present bias. Naiveté may lead individuals to delay high return investments that have a short-run utility cost (even if small) because they incorrectly anticipate making these investments later.

Time preferences are typically elicited by asking participants to choose between a smaller-sooner option that gives them a relatively small financial reward relatively soon and a larger-later option that gives them a larger reward after a longer delay. 6 These experiments are often conducted in conjunction with risk experiments. For example, Tanaka et al. ( 2010 ) conduct incentivized time and risk preference experiments in rural Vietnam and find that in villages with higher mean income, people are less loss-averse and more patient. This shows that household income is correlated with patience but not with risk.

Pro-social preferences : Measuring pro-social preferences using field experiments is common. Tasks to elicit pro-social preferences include dictator games as a measure of altruism, public goods games to measure cooperation, trust games to measure trust and trustworthiness, and ultimatum games to measure fairness in bargaining. Social preference tasks have also been used to investigate cross-country differences in behaviour and subject pool (Henrich, 2000 and Henrich et al., 2001 ), in-group and out-group biases across societies (Afridi et al., 2015 ; Chen & Li, 2009 ; Gupta et al., 2018 ), patterns of behaviour across matrilineal and patriarchal societies (Andersen et al., 2013 ; Banerjee et al., 2015a , 2015b ; Gangadharan et al., 2021 ; Gneezy et al., 2009 ; Mukherjee, 2018 ), and patterns of evolution by examining the behaviour of school children and adolescents (Harbaugh & Krause, 2000 ; Sutter et al., 2013 ).

Anti-social preferences : Researchers have also examined anti-social behaviour using lab-in-the-field experiments (e.g. Prediger et al. ( 2014 ) and Gangadharan et al . ( 2020 ), explore anti-social preferences of pastoralists in Namibia facing resource scarcity and individuals exposed to the Cambodian genocide, respectively). A voluminous literature uses the die-tossing task (Fischbacher & Föllmi-Heusi, 2013 ) or variants of it (where participant earnings depend on self-reported outcomes) as a reliable measure of dishonesty and corruption at both the individual and macroeconomic level. Several studies have used such tasks to examine dishonesty amongst different groups of the population. These include bankers (Cohn et al. ( 2014 )), prisoners (Cohn et al. ( 2015 )), public and private sector aspirants (Banerjee et al., ( 2015a , 2015b )), civil servants and nurses (Hanna and Wang ( 2017 )), students and teachers (Cohn and Maréchal ( 2019 )), milk vendors (Kröll and Rustagi ( 2020 )) and elected politicians (Chaudhuri et al ., 2020 , 2021 ). Gächter and Schulz ( 2016 ) and Olsen et al. ( 2019 ) have conducted cross-sectional studies to examine how behaviour in the die-tossing task is correlated with country-level measures of corruption such as the Corruptions Perceptions Index. They find that citizens of more (less) corrupt countries tend to be less (more) truthful in reporting their outcomes in the die-tossing task.

Eliciting beliefs : Since beliefs can have a large influence on decisions, eliciting beliefs about behaviour within an experiment is increasingly common. For example, whether an individual contributes to a public good is in part influenced by whether they believe others will also contribute. While a useful summary of elicitation methods can be found in Schotter and Trevino ( 2014 ) and Schlag et al. ( 2015 ), the core idea is to elicit from participants what they believe occurred (e.g. “how much did you think your group members would contribute in the public goods game?”) with financial incentives based on how accurate a participant’s belief is with respect to the actual behaviour. Bursztyn et al. ( 2020 ) report an interesting application in relation to beliefs by studying women working in the labour force in Saudi Arabia.

Social norms : While the inclusion of social norms within economic models is relatively new, research on social norms has produced significant evidence revealing their importance in understanding behavioural mechanisms (Gangadharan et al., 2016 ; Dimant et al., 2020 ; Jayachandran, 2020 ). A common method to measure social norms is to use a coordination game (Krupka and Weber, ( 2013 )), where participants are asked to guess what they think others believe is socially appropriate in a given context. Responses are often given on a four-point scale ranging from very socially appropriate to very socially inappropriate. Unlike standard belief questions, these questions specifically focus on the social appropriateness of behaviour (for instance, respondents could be asked whether it is socially appropriate for a male to work as a homemaker). This method is useful because subjects have an incentive to reveal what they think others believe is socially appropriate, not what they believe themselves (Exley et al., 2020 ; Gächter et al., 2013 ). Varying the reference group is possible, which allows researchers to elicit what respondents think females believe as compared to what males believe. We use this method in the research described in Sect.  4 .

Does the gender of the decision-maker matter?

Gender and leadership: impact of affirmative action.

In this section, we describe lab-in-the-field research focusing on gender and leadership, especially that reported in Gangadharan et al. ( 2016 ) and Gangadharan et al. ( 2019 ), which illustrate how lab-in-the-field experiments can be combined with a natural policy change to provide insights into behaviours that can undermine the leadership of women.

Research in this area has grown rapidly as a response to the increasing concern about the poor representation of women in leadership positions. Women constitute only 7% of all heads of government, 4.8% of Fortune 500 company CEOs, 7% of central bank governors and 2.5% of self-made billionaires. When given the opportunity, women leaders often make different policy decisions compared to men. This has been shown in terms of workforce reductions (Adams & Ferreira, 2009 ; Ahern & Dittmar, 2012 ; Matsa & Miller, 2013 ; Eckbo et al . 2019; Bertrand et al., 2019 ), spending on vulnerable populations (Chattopadhyay & Duflo, 2004a , 2004b ; Edlund & Pande, 2001 ; Lott & Kenny, 1999 ; Pande & Ford, 2012 ) and corruption (Brollo & Troiano, 2016 ). Together, this suggests that more women in leadership positions could offer substantial economic and social benefits. A prominent approach to increasing the number of women in leadership positions has been to use affirmative action policies, such as quotas or reservation of seats for women in leadership positions.

Observational data from developed countries on the effect of gender quotas on female political engagement provides mixed evidence. In support of this measure, O'Brien and Rickne ( 2016 ) find that a quota for women introduced in Sweden increased the number of women perceived as qualified for higher positions and accelerated women's representation in leadership positions, while De Paola et al. ( 2010 ) and Baltrunaite et al . ( 2014 ) show that even temporary quotas increased women's representation in politics and led to higher quality politicians (both men and women) being elected to office. Conversely, Bagues and Campa ( 2020 ) and Lassebie ( 2020 ) find no evidence that quotas in Spain and France led to systematic improvements in the career progression of women.

Ultimately, the likelihood of women becoming successful leaders depends on whether citizens (both men and women) accept them in leadership positions. However, little is known about how men and women respond to female leaders. The behavioral response towards and of leaders can be a difficult question to answer using observational data. We thus investigate this using two lab-in-the-field experiments, which present some unique advantages in this context. 7 First, the allocation of leader roles in the experiments enables data to be collected from female decisions makers, which is often not feasible using observational data due to the limited number of women in leadership roles. Second, the randomized assignment of leadership status avoids selection issues relating to the leader’s identity, meaning that the citizens’ actions in response to the leader’s gender can be interpreted as causal. This is difficult using observational data, as other leader characteristics can confound how the leaders are perceived. Third, with observational data there is no straightforward way of disentangling the preferences and reactions of the leaders and the citizens. However, experiments can measure the decisions of both leaders and citizens and separately attribute their decisions to the treatment they are in. Fourth, the experimental approach reduces social desirability bias (as noted in Sect.  3 ). Finally, the field context allows the experimental findings to be placed in the context of participants’ actual exposure to male and female leaders in their villages.

The first experiment is a leadership experiment conducted in 40 villages, with 956 participants, in three districts in the state of Bihar in India. The design utilises changes introduced by the 73rd Amendment to the Indian Constitution, ratified by the National Parliament in April 1993, which codified a system of rural local governance (called the “Panchayati Raj”) and also introduced quotas for women in every level of rural local governance. In each state, at least one third of all seats in every village, sub-district or district level council were to be reserved for women, and across the state at least one third of all head of village council positions were to be reserved for women. 8 Bihar introduced the Panchayati Raj system later than many other states, and is categorized as a late adopter of the 73rd Amendment to the Indian constitution; the first Panchayati Raj election in Bihar was held in 2001. However, Bihar was the first state to increase the proportion of seats to be reserved for women to 50%, a norm later adopted by most states. In 2014, when these experimental sessions were run, Bihar had participated in three Panchayati Raj (including village council) elections: in 2001, 2006 and 2011.

We use this policy, and the results from the elections, to define female-headed and male-headed villages. We classify a village as a female-headed village if it had at least one female head in the three village council elections, and a male-headed village if it never had a female head. Those living in male-headed villages have very limited experience with women as leaders. 9

The leadership experiment is a modified one-shot linear public goods game. Participants are randomly assigned to groups. Each group consists of two men and two women. Group composition is public information. One group member is randomly and anonymously chosen as the group leader; the remaining three group members can be thought of as citizens. There are multiple groups in each session. Participants are not provided any information about the identity of the other members of the group, apart from the gender composition of the other group members. Notably, half the groups in each session have female leaders. As in a standard public goods game, each participant receives an endowment (in this case, Rs. 200). The game has two stages. In the first stage, the leader proposes a non-binding contribution ( p i ) towards the group account. The leader’s proposal is communicated anonymously to group members. In the second stage, all group members, including the leader, simultaneously contribute towards the group account, with payoffs calculated as Π = c i + α ∑ i g _ i 4 ; c i + g i = 200 ; α = 2 . Here, g i is the amount that individual members contribute to the public account, and c i is what is left for their private account. The leader’s proposal is not binding (cheap talk) so in the second stage they are not restricted to contributing what they had proposed in the first stage of the experiment i.e., p i > = < g i .

We conduct two treatments with an approximately equal number of participants. Gender revealed where the group members are informed of the leader’s proposed amount and the leader’s gender and Gender not revealed , where the group members are only informed about the leader’s proposed amount. The treatments allow us to observe the behaviour of male and female members towards male and female leaders. Further, we can exogenously vary the observability of the leader’s identity, allowing us to understand how citizens will behave when the leaders gender is unknown. 10 Both these aspects are challenging to implement using other empirical approaches.

We, therefore, have two sources of random variation: (1) treatment variation arising from the experimental design (variation in the information about the gender of the head of the group, i.e., male head, female head or the head of the group is not known); and (2) exogenous variation in exposure to female heads of village councils, introduced through the affirmative action policy. 11

Figure  3 depicts the key difference estimates reflecting the additional contribution made by male citizens and female citizens in female-led groups (relative to male-led groups), computed using a multivariate regression analysis. We find that male citizens contribute Rs. 13 (or 7% of their endowment) less in female-led groups relative to male-led groups, while female citizens’ contributions show little variation based on the leader’s gender. We refer to this as male backlash against female group leaders.

Additional Contribution in Female Led Groups, by Male and Female Citizens. The bars denote the additional contribution in female-led groups by male and female citizens (a negative amount is shown if the contribution is higher in male-led groups). The capped lines denote the 90% confidence intervals. For further details, see Table 4 in Gangadharan et al. ( 2016 )

To understand whether these differences differ by the gender of the head of the village council, we compare behaviour in female-led groups in both male and female-headed villages. Figure  4 presents the difference estimates diagrammatically. We find that male citizens contribute significantly less to female led groups (relative to male led groups), but only in female-headed villages. The estimated effects imply that men contribute Rs. 24 (or 12% of their endowment) less to female-led groups in female-headed villages. Irrespective of whether the village is male headed or female headed, female citizens do not contribute differentially in male and female-led groups. This reveals that the male backlash against female leaders is driven by a male response against female leaders in female-headed villages.

Additional Contribution in Female Led Groups by Male and Female Citizens in Male and Female-Headed Villages. The bars denote the additional contribution in female-led groups by male and female citizens in male and female-headed villages (a negative amount is shown if the contribution is higher in male-led groups). The capped lines denote the 90% confidence intervals. For further details, see Gangadharan et al. ( 2016 )

To better understand the mechanisms driving our key results (Figs.  3 and ​ and4), 4 ), we conducted a second experiment. This was a social norms experiment. It was conducted in 21 villages with 267 individuals. The participants for the social norms experiment lived in the same districts (but not in the same villages) as those who participated in the leadership experiment. Participants undertook three tasks that described possible decisions made by participants in the original leadership experiment as well as a number of vignettes. 12 Participants in the social norms experiment rated each decision along a four-point Likert scale (very socially inappropriate, somewhat socially inappropriate, somewhat socially appropriate and very socially appropriate). Incentivization encouraged participants to reveal their beliefs about others’ actions rather than their own preferences. The participants’ aim was to match their responses with those of others (similar to a coordination game). As discussed in Sect.  3.1 , the use of incentivization distinguishes this experiment from a survey measure that elicits norms, and increases the cost of responding in a socially desirable manner.

To understand the motivations for the male backlash observed in the leadership experiment, we focus on responses in one specific task in the social norms experiment, namely that asking subjects to rate the social appropriateness of a male/female citizen contributing Rs. 0, 50, 100, 150 or 200 to the group account, when the group leader is male or female. We find that males believe that it is more socially appropriate for men to contribute 50% or less of their endowment to a female-led group (i.e., it is socially less costly for males to contribute less to female-led groups). This pattern of expectations about the behaviour of male and female citizens is consistent with the observed backlash against female leaders.

However, this is not the only explanation for such a backlash. A second potential driver is the identity crisis that men may face when women are appointed to leadership positions. Males may view leadership and power as the domain of men, and may experience a loss of identity when women encroach upon this domain. Gender is a particularly strong aspect of identity, and the appointment of women in a leadership position may lead to a violation of male identity. Individuals (in this case, men) who believe their identity is being violated might act out to bolster a sense of self or to salvage a diminished self-image. Consistent with this notion, male identity is more conspicuous in female-headed villages, which is where men’s role is being challenged. As Fig.  4 documents, backlash against female leaders is being driven by the behaviour of men in female-headed villages.

What about the leaders themselves? We argue that the social environment is another potential barrier to the effectiveness of women leaders. This is because women leaders may respond negatively to the expectation that men are likely to react differently to them (especially those in authority due to an affirmative action policy). Such a response could reduce trust and cooperation, and consequently decrease social and economic welfare in the long run, further diluting the effectiveness of women’s leadership.

Accordingly, we consider whether there is a gender difference in leaders’ propensity to contribute less to the public good than stated in their proposals. The results presented in Fig.  5 , Panel A show that female leaders are, on average, 20 percentage points more likely to choose to deviate negatively from their stated proposals compared to male leaders (i.e., g i < p i ). However, a related and potentially more important question is whether these differences in behaviour reflect differences in male and female leaders’ inherent preferences, or whether they reflect differential responses by male and female leaders to the social environment. This is important because policy prescriptions may differ depending on the cause of the behaviour. To examine whether the social environment is driving the results, we first consider deviations by leaders depending on whether the gender of the group leader is revealed to the citizens (the treatment effect). The results are presented in Fig.  5 , Panel B. They indicate that female group leaders are 23.5 percentage points more likely to deviate negatively in gender revealed sessions, and only 16.3 percentage points more in gender not revealed sessions.

Gender Differences in the likelihood of Negative Deviation. The bars denote the additional likelihood of negative deviation ( p i - g i ) by women leaders (positive if women deviate more and negative if men deviate more). The capped lines denote the 90% confidence intervals. For further details, see Gangadharan et al. ( 2019 )

Second, we consider the influence of the gender of the village head. The likelihood of negative deviation in female and male-headed villages is presented in Fig.  5 , Panel C. Compared to male leaders, female leaders are significantly more likely to exhibit negative deviation in female-headed villages. The likelihood of observing a negative deviation is not statistically different for male and female leaders in male-headed villages.

Taken together, these results suggest that, when the social environment is such that women anticipate men in their group will not cooperate (either because the men know the gender of the group leader or because they are in female-headed villages), they react negatively by reducing their contributions as compared to their proposals.

Further explorations of the social environment

One way in which researchers have examined the impact of the social environment on gender is to conduct lab-in-the-field experiments in locations with different cultural and historical gender-related institutions. An example of this is Gneezy et al. ( 2009 )’s utilisation of lab-in-the-field experiments to explore the underpinnings of gender differences in competitive attitudes across the Maasai in Tanzania (a patriarchal society) and the Khasi in India (a matrilineal society). They find that, among the Maasai, men opt to compete at roughly twice the rate of women, but among the Khasi this phenomenon is reversed. This finding suggests that nurture may be a stronger determinant of competitive inclination than nature. Andersen et al. ( 2013 ) extend this line of argument by investigating whether women are born less competitive than men, or whether they become so through the process of socialization, by comparing the competitiveness of children in matrilineal and patriarchal societies. They find no gender differences at any age in the matrilineal society, but that girls become less competitive around puberty in the patriarchal society. Researchers have also investigated gender differences in negotiation (Andersen et al., 2018 ), public good provision (Andersen et al., 2008 ; Banerjee et al., 2015a , 2015b ), investment in risky microfinance projects (Mukherjee, 2018 ), and dictator game giving (Gong et al., 2015 ) in matrilineal and patriarchal societies.

Building on this body of research, we aim to better understand the role of the social environment in the process of decision-making. As reported in the previous section, we found that male and female leaders are treated differently by group members. In Gangadharan et al. ( 2021 ), we aim to further explore this issue by studying whether the choices of male and female decision-makers are evaluated differently by independent third parties (i.e. participants who, unlike a group member, are not monetarily affected by the actions of the decision-maker). As Fehr and Fischbacher ( 2004 ) argue, unaffected third parties, by their willingness to incur a cost to sanction decision-makers for the transgression of norms, reveal the true normative standards of behaviour. Such third-party evaluations are universal across cultures and across time (see Greif, 1993 , 1994 ; Henrich et al., 2006 ).

The key research questions we examine are: (i) whether male and female third parties evaluate the actions of male and female decision-makers differently; and (ii) whether these evaluations vary across social environments. We conduct two lab-in-the-field experiments. Both involve a variant of a third-party dictator game (Fehr & Fischbacher, 2004 ), consisting of two stages. In Stage 1, which is similar to a standard dictator game, a decision-maker (a “Proposer”) chooses how to divide an endowment between themselves and a passive recipient. In Stage 2, a third party (the “Judge”) evaluates the choice and can choose to punish the Proposer for their actions. Punishment is costly, but by investing π the Judge can increase or decrease the Proposer’s income by 3 π . If the salient norm is one of equal distribution, theories of social preference would suggest that Judges care about the norm of equality and may choose to punish transgressions of this norm. We investigate whether there are gender differences in punishments and whether these are influenced by the social environment.

The variation in the social environment is captured by conducting the two experiments in regions of India where norms relating to gender differ widely. Experiment 1 is conducted with matrilineal societies in the state of Meghalaya. Experiment 2 is conducted in the state of Haryana, which is characterised by strong patriarchy, regressive gender norms and strong gender bias against women. We find that the social environment can be critical in how men and women are evaluated for transgressing norms. In a more gender-equal environment (Experiment 1), both male and female decision-makers are punished. However, men are more likely to punish male decision-makers, while women are more likely to sanction female decision-makers, for transgressions. In an environment with significant gender biases (Experiment 2), female decision-makers are significantly more likely to be punished than men for the same transgression. In this case, however, male evaluators are more likely to punish female transgressions. We infer that male-dominated environments such as that in Experiment 2 can be more susceptible to stereotypes (e.g. that women are less selfish than men) so, for the same transgression, women are punished more heavily when they act against the stereotype.

These results suggest that the social environment is crucial for understanding leaders’, decision makers’ and evaluators’ patterns of behavior. Lab-in-the-field experiments facilitate insights into this area by allowing researchers to control information and action space (meaning that male and female decision-makers can be assessed based on similar actions), enabling causal inferences relating to the gender of the decision-maker to be drawn.

Challenges of lab-in-the-field experiments

A number of ethical and practical challenges emerge when conducting lab experiments in the field. In this section, we reflect on our experience conducting the experiments reported in Sect.  4 to offer some insights into the challenges we faced and the strategies we adopted in response to them.

Ethical concerns and ethics clearance

Primary data collection requires extensive safeguards to protect the human subjects participating in the research. This is especially true of experiments conducted in developing countries which present different, and potentially more serious, ethical issues than studies run in more developed countries, in part due to the majority of participants being poor, illiterate, unable to provide consent and having limited access to formal legal processes. While some of these concerns apply to any field-based economics research involving disadvantaged and vulnerable communities, others are specific to laboratory experiments conducted in the field.

A particular issue to consider is how ethical clearance is obtained. Where lab-in-the-field experiments are conducted in the country where the researchers are located, this is straightforward, as it is sufficient for researchers to approach the ethical review boards at their home institutions. However, lab experiments conducted internationally present additional challenges for review boards in developed countries (Krogstad et al., 2010 ). As a consequence, lab-in-the-field experiments conducted in a developing country may also require ethics clearance from that country. In many cases, co-authors on the project will be located and employed in the country where the experiment is scheduled, and their institution may insist on research oversight. This was the primary reason for also seeking ethics clearance from the Indian School of Business on our project. Local review boards may also have insights into research context and subject protection that external boards lack. However, local review boards may also be unfamiliar with the processes and standards involved in field experiments in economics, and may independently decide which consent process is required, or rely on standards from other disciplines (such as psychology or medicine) which unduly inconvenience or prevent the research. Navigating this process can be difficult and time consuming for researchers, and this should be kept in mind when planning projects.

Ethics of payment : Field researchers have grappled with the ethics of paying subjects for participating in research (Grady, 2019 ). However, the offer of payment may also influence subjects’ decision to participate in the research. 13

In the field of experimental economics, financial payment for participation and decision-making is a well-established practice (Smith, 1976 ). Payments reimburse subjects for out-of-pocket expenses (such as travel to the experimental site), compensate them for their time and the burdens of participation, incentivize participation (Gelinas et al., 2018 ) and make choices in the experiment salient, mirroring decisions made outside the lab (Smith, 1976 ). In the case of lab experiments conducted in developing countries, the payments present several challenges. Foremost is what the level of payment should be. Few incentivized experiments are conducted in developing countries, so there are not many benchmarks to guide researchers. 14 In our experiment, we aimed at average earnings of approximately Rs. 400 (USD 6 under current exchange rates) per participant for approximately 2–3 h of work. This corresponded to two days’ wages for unskilled workers in rural areas of Bihar.

Relatively high payments can lead to high turn up rates. In cases where more people want to participate in a session than are required, a transparent selection mechanism is necessary (similar to those used to decide which participants to turn away in the event of over-subscription to a lab experiment). For example, a public lottery could be used to determine participation.

In addition, even within countries, researchers need to vary earnings by local economic conditions (Charness et al., 2013 ). For instance, show-up fees to ensure participation among agricultural workers might be very different from those offered to office workers. This is because, if the researcher offered a uniform rate, it might, for example, influence the types of office workers who show up in a way that impacts the experimental findings (e.g. they might be unemployed or particularly enthusiastic about participating in research). Again, there are few benchmarks to guide such decisions. Our experiments were conducted in villages with similar economic conditions. All our sessions were conducted during the agricultural lean season when there is an excess supply of labour, ensuring that the opportunity cost of time and the effective payment rate were similar across locations.

Sample selection and recruitment : Sample selection is challenging for all primary data collection work in economics, but the hurdles are greater for experiments conducted in the field. Numerous researchers have highlighted the challenges of representative samples in university laboratory settings, and how these might be amplified in field experiments (see Cardenas and Carpenter ( 2008 ) and Harrison and List ( 2004 )).

With respect to the first experiment we conducted in Bihar, our budget and time constraints meant that, at best, we could conduct 40 sessions with 960 participants in only a few locations. For a state with over 100 million inhabitants, this sample could not constitute a representative one. As a consequence, we could not fully claim sample representativeness. Instead, we focused on the internal validity of the experiment with more limited claims exploiting within experiment variation in treatment status. In addition, we used administrative census data to make informed statements about the representativeness of the sample. Given that our objective was to investigate men and women’s reactions to male and female leaders, having an equal number of men and women participate in the experiment was essential. While enrolling men was easy, targeting women was more challenging given the prevailing social norms and restrictions on women’s outside interactions and movements. 15

To address this challenge, our recruitment strategy was to ensure that our research team included a number of female research assistants who could speak the local language and go door-to-door informing women of the opportunity to participate. We also distributed flyers that contained information about the eligibility criteria, the remuneration range and the time and location of the sessions. These flyers were also posted at prominent villages landmarks (e.g., post office, bus stops, schools, teashops, places of worship). Another possible strategy is to use reliable local agents (paid or unpaid) who can help select participants according to criteria determined by the researchers. 16  We did not use agents as several of our research assistants were from Bihar.

Adapting instructions : Lab-in the-field experiments conducted across cultures can often involve languages other than English, and therefore require translation since most economics research is communicated in English. Roth et al. ( 1991 ) discuss several concerns associated with language translation that emerged while conducting experiments in Israel, Japan, Slovenia and the USA. As researchers conducting experiments in India, our protocols were developed in English, with final results also written in English. However, almost no participant was fluent in English, and therefore the experiments were conducted in the dialect of Hindi that was most prevalent in Bihar. We decided to recruit research assistants with spoken and written fluency in the local Hindi dialect as well as a background in economics so that they understood the research questions and respected the demands of the experimental methodology, but were also attuned to the local social and linguistic nuances. To prepare the translated materials, the research assistants suggested Hindi terms for technical economics ideas during our initial discussions. Then one team of research assistants translated the instructions and questionnaires into Hindi, while a second team translated the Hindi version back into English. Insofar as the original English version matched the re-translated version, we could be confident that the Hindi version was a faithful translation of the original. Three out of four of the researchers were also fluent in Hindi, and independently verified the translation.

While piloting the experiment, we discovered that despite being literate, many participants were not practiced readers and writers, and therefore pictorial instructions were critical. 17 The instructions were read slowly and deliberately, and included several examples that showed how experimental choices translated into earnings. We included pauses where participants could ask clarifying questions. All experiments were conducted using paper and pencil because participants were familiar using these. Laptops were both unfamiliar to participants, and transporting, charging and storing them was a logistical challenge. One downside to paper-based experiments was that each experiment took longer to conduct than it would have with digitized data collection.

Experimenters : Field experiments require more research personnel, and people with different skills than those needed in lab settings. Reflecting this, Derry and Baum ( 1994 ) discuss the variety of situations experimenters in field settings may encounter, and how trained experimenters learn to anticipate and deal with these situations. Our experiments required us to cover 40 villages (with one day of canvasing and one day of experiments in each village) with appropriate periods for rest in one month. Given this time constraint, we created two teams, each of which included an experimenter and male and female research assistants for canvasing and conducting post-experiment surveys. The research assistants visited the experiment site the day before the experiment was scheduled to ensure that the venue of the session was satisfactory, and to recruit participants. The two teams operated in parallel to complete the experiments in time.

The research assistants were primarily recruited from Masters of economics programs in Patna, Bihar. While they were trained in microeconomics, they had little exposure to experimental economics. To develop their skills, the researchers held a 3-day training program with: (i) conceptual training on the value of the experimental method in economics; (ii) detailed training on the specifics of the experiment we planned to conduct, including mock experiments; and (iii) pilot testing conducted with recruited participants in a village outside Patna.

The safety of the research team and participants was paramount. The team carried expensive laptops, mobile phones and tablets, and a significant amount of cash to pay the experimental participants. For this reason, we started and finished early, so that the research team could return to their base before nightfall. In 2020, the spread of COVID-19 and the lockdowns imposed by governments added another layer of complexity. However, we adopted the principle of safety first, and (temporarily) discontinued field work.

In this paper, we discuss the role played by lab-in-the-field experiments in economics, which the literature also refers to as artefactual, framed and extra-lab experiments. These methods have been used in many areas of economics and have provided researchers significant insights into social and economic phenomena, by focusing on outcomes, mediators (mechanisms) and moderators (heterogeneity). We present examples of how lab-in-the-field experiments can be conducted on their own or combined with conventional lab experiments, natural experiments, RCTs and surveys. We also highlight some of the unique opportunities and challenges presented by this approach, drawing on our recent research on gender and leadership as an example.

We anticipate two key avenues for growth in this area. In recent years, the use of online experiments and experiments embedded in household surveys has increased, and this is expected to continue. This allows researchers to investigate questions using a different lens and with a diverse sample. Another direction of growth may be in the increased use of technology, both for conducting experiments (using apps ) and for recruiting participants (using GIS software).

1 All experiments, including conventional laboratory experiments, involve random assignment of participants to one of the treatments. We follow the literature in economics, where RCTs are used to refer to experiments in the field where the intervention is done by researchers or policy makers, with participants who are familiar with the field context and requires them to be engaged to some degree (e.g., via providing survey responses).

2 There are several excellent review articles that focus on the use of field experiments in specific areas. Examples include Bertrand and Duflo ( 2016 ) on discrimination, List and Price ( 2016 ), Price ( 2014 ) and Brent et al. ( 2017 ) on environmental economics, and List and Rasul ( 2011 ) on labour economics.

3 Differences in behaviour across subject pools has been of considerable methodological interest to experimental economists (see for example, Ball and Cech ( 1996 ), Frechette ( 2011 ) and Snowberg and Yariv ( 2021 )).

4 RCTs are usually designed to ensure balance amongst observable variables across treatment and control groups, and researchers use the baseline surveys to check for this balance. Lab-in-the-field experiments can help in opening the black box referred to as un-observable variables.

5 Falk and Heckman ( 2009 ), Friedman and Sunder ( 1994 ), Reuben and Winden ( 2008 ) and Xiao and Houser ( 2005 ) demonstrate this using lab experiments.

6 While many experiments have identified preferences using time-dated monetary payments, identifying time preferences from such monetary choices has also been argued to be problematic (Cubitt and Read ( 2007 )). To address this, Augenblick et al. ( 2015 ) investigate choices over consumption (real effort), in addition to time-dated financial rewards, and find significant evidence of present bias in the domain of effort, but not in the domain of money. In particular, subjects allocate more work to the first work date, when the allocation of tasks is made in advance as compared to when it is made on the first work date itself. They also document a relationship between present bias and the demand for commitment as those who are potentially aware of their present bias, take actions to limit their future behaviour.

7 The literature using experiments (summarised in a recent survey by Eckel et al. ( 2021 )) has shown that both preferences and beliefs play a critical role in explaining the gender gap in leadership.

8 Note that when a seat is reserved for women, only women can stand as candidates. However, all electors (male and female) are eligible to vote in the elections. Additionally, while quotas for women were introduced at the lower levels of governance, there were no corresponding quotas for women in state assembly and national parliament elections.

9 Although, strictly speaking, a female headed village should denote a village where the position of the head of the village council is reserved for women, we use the two meanings interchangeably because, while it is theoretically possible for a woman to be elected to the position of village head in the absence of a quota, this almost never happens in practice.

10 In both treatments, at every stage decision sheets are handed out and collected simultaneously from all group participants, helping preserve the anonymity of all participants. The leader’s decision and gender are only revealed to the members through the decision sheets. In the Gender revealed sessions, gender was (subtly) revealed using gender icons for the leader.

11 For more details see Gangadharan et al. ( 2016 ) and Gangadharan et al. ( 2019 ).

12 Participants had to rate the behavior of group members (both male and female) contributing 0 through to 200 in intervals of 50. Responses were elicited for both male and female led groups. Participants were also asked to rate the social appropriateness of a leader (both male and female) contributing a certain amount (Rs. 50/100/150) when they propose 100 and 200.

13 These challenges have been recognized in the context of biomedical research, raising “ethical concerns, such as undue inducement, exploitation, and biased enrollment” (Resnick ( 2015 )).

14 A useful resource on this issue is Cardenas and Carpenter ( 2008 ).

15 Recruiting participants who are unable to provide consent (e.g., children) can present additional challenges.

16 For example, Cameron et al. ( 2013 ) employed a survey company that used an advertisement designed by the researchers to recruit participants through their regular subject-recruitment-network. The company also approached people on the street, and posted the advertisement on their website, notice boards, and street lampposts across the city. Similarly, Chaudhuri et al . ( 2020 ) and Chaudhuri et al. ( 2021 ) use a survey company that contacted the council officials and identified elected local politicians to find the appropriate sample for their research. Gangadharan et al . ( 2020 ) obtained survey contact lists from the local government statistical organization. They also hired local research assistants who received help from teachers, principals and village elders who were familiar with the demographic distribution of their communities.

17 Cardenas and Carpenter ( 2008 ) raise issues of language sophistication and the cognitive ability of participants, while also arguing in favour of pen-and-paper experiments that are easier to administer.

We thank Kajal Agarwal, Xialene Chang, Sarah Meehan and Vishal Pant for outstanding research assistance. The Editors, an anonymous reviewer, Utteeyo Dasgupta, Paulo Santos and participants at the 2021 International Workshop for Lab and Field Experiments, organised by the Japanese Economic Association, provided useful comments and suggestions. Research reported in this paper received funding from IGC-Bihar, Monash University, the Indian School of Business, the Australian Research Council (DP1411900, DP170101167), the Australian Government’s Endeavour Research Fellowship, and the BMGF/UC San Diego EMERGE program.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Contributor Information

Lata Gangadharan, Email: [email protected] .

Tarun Jain, Email: ni.ca.amii@jnurat .

Pushkar Maitra, Email: [email protected] .

Joe Vecci, Email: [email protected] .

• Adams R, Ferreira D. Women in the Boardroom and Their Impact on Governance and Performance. Journal of Financial Economics. 2009; 94 :291–309. [ Google Scholar ]
• Afridi F, Li SX, Ren Y. Social Identity and Inequality: the impact of China's Hukou system. Journal of Public Economics. 2015; 123 :17–29. [ Google Scholar ]
• Ahern K, Dittmar A. The changing of the boards: the impact on firm valuation of mandated female board representation. Quarterly Journal of Economics. 2012; 127 :137–197. [ Google Scholar ]
• Alatas V, Cameron L, Chaudhuri A, Erkal N, Gangadharan L. Subject pool effects in a corruption experiment: a comparison of indonesian public servants and indonesian students. Experimental Economics. 2009; 12 (1):113–132. [ Google Scholar ]
• Andersen S, Bulte E, Gneezy U, List JA. Do women supply more public goods than men? Preliminary experimental evidence from matrilineal and patriarchal societies. American Economic Review: Papers and Proceedings. 2008; 98 :376–381. [ Google Scholar ]
• Andersen S, Ertac S, Gneezy U, List J, Maximiano S. Gender, competitiveness and socialization at a young age: Evidence from a matrilineal and a patriarchal society. Review of Economics and Statistics. 2013; 95 :24–30. [ Google Scholar ]
• Andersen S, Ertac S, Gneezy U, List J, Maximiano S. On the cultural basis of gender differences in nesgotiation. Experimental Economics. 2018; 21 :757–778. [ Google Scholar ]
• Augenblick N, Niederle M, Sprenger C. Working over time: dynamic inconsistency in real effort tasks. Quarterly Journal of Economics. 2015; 130 :1067–1115. [ Google Scholar ]
• Bagues, M. F., & Campa, P. (2020). Can gender quotas in candidate lists empower women? Evidence from a regression discontinuity design . CEPR Discussion Paper No. DP12149.
• Ball SB, Cech PA. Subject pool choice and treatment effects in economic laboratory research. In: Greenwich CT, editor. Research in experimental economics. Greenwich CT: JAI Press; 1996. pp. 239–292. [ Google Scholar ]
• Baltrunaite, A., Casarico, A., & Profeta, P. (2014). Spill-over effects of affirmative action: political representation and the power of the elderly . CESIFO Working Paper No. 4955.
• Banerjee, D., Ibañez, M., Riener, G., & Wollni, M. (2015a). Volunteering to take on power: Experimental evidence from Matrilineal and Patriarchal societies in India . DICE Discussion Paper, No. 204. Düsseldorf: Düsseldorf Institute for Competition Economics (DICE).
• Banerjee R, Baul T, Rosenblat T. On self-selection of the corrupt into the public sector. Economics Letters. 2015; 127 :43–46. [ Google Scholar ]
• Banerjee, P., Iversen, V., Mitra, S., Nicolo, A., & Sen, K. (2020). Moral reputation and political selection in a decentralized democracy: theory and evidence from India . UNU WIDER Working Paper 2020/26.
• Barr A, Lindelow M, Serneels P. Corruption in public service delivery: an experimental analysis. Journal of Economic Behavior & Organization. 2009; 72 :225–239. [ Google Scholar ]
• Barrett C, Carter M. The power and pitfalls of experiments in development economics: some non-random reflections. Applied Economic Perspectives and Policy. 2010; 32 :515–548. [ Google Scholar ]
• Bartling B, Fehr E, Marechal MA, Schunk D. Egalitarianism and competitiveness. American Economic Review. 2009; 99 :93–98. [ Google Scholar ]
• Beath A, BenYishay A, D’Adda G, Grosjean P, Weber R. Can vouchers reduce elite capture of local development projects? Experimental evidence from the Solomon Islands. Journal of Public Economics. 2018; 130 :117–131. [ Google Scholar ]
• Belzil, C., & Leonardi, M. (2009). Risk aversion and schooling decisions . Working Paper # 2009–028. Ecole Polytechnique.
• Bertrand M, Black SE, Jensen S, Lleras-Muney A. Breaking the glass ceiling? The effect of board quotas on female labour market outcomes in Norway. Review of Economic Studies. 2019; 86 :191–239. [ Google Scholar ]
• Bertrand M, Duflo E. Field experiments on discrimination. In: Banerjee A, Duflo E, editors. Handbook of economic field experiments. Elsevier; 2016. [ Google Scholar ]
• Bhalotra, S., Clots-Figueras, I., Iyer, L., & Vecci, J. (2021). Leader identity and coordination. Review of Economics and Statistics (forthcoming).
• Binswanger HP. Attitudes toward risk: experimental measurement in rural India. American Journal of Agricultural Economics. 1980; 62 :395–407. [ Google Scholar ]
• Blackburn M, Harrison G, Rutström E. Statistical bias functions and informative hypothetical surveys. American Journal of Agricultural Economics. 1994; 76 :1084–1088. [ Google Scholar ]
• Blair G, Imai K. Statistical analysis of list experiments. Political Analysis. 2012; 20 :47–77. [ Google Scholar ]
• Brent DA, Friesen L, Gangadharan L, Leibbrandt A. Behavioral insights from field experiments in environmental economics. International Review of Environmental and Resource Economics. 2017; 10 :95–143. [ Google Scholar ]
• Brollo F, Troiano U. What happens when a woman wins an election? Evidence from close races in Brazil. Journal of Development Economics. 2016; 122 :28–45. [ Google Scholar ]
• Brown P, Daigneault AJ, Tjernström E, Zou W. Natural disasters, social protection, and risk perceptions. World Development. 2018; 104 :310–325. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Bursztyn L, González AL, Yanagizawa-Drott D. Misperceived social norms: women working outside the home in Saudi Arabia. American Economic Review. 2020; 110 :2997–3029. [ Google Scholar ]
• Cadsby CB, Maynes E. Choosing between a socially efficient and free-riding equilibrium: Nurses versus economics and business students. Journal of Economic Behavior & Organization. 1998; 37 :183–192. [ Google Scholar ]
• Camerer CF, Hogarth RM. The effects of financial incentives in experiments: a review and capital-labor-production framework. Journal of Risk and Uncertainty. 1999; 19 :7–42. [ Google Scholar ]
• Cameron L, Erkal N, Gangadharan L, Meng X. Little emperors: behavioural impacts of china’s one-child policy. Science. 2013; 339 :953–957. [ PubMed ] [ Google Scholar ]
• Cameron L, Meng X, Zhang D. China’s sex ratio and crime: behavioral change or financial necessity. Economic Journal. 2019; 129 :790–820. [ Google Scholar ]
• Cameron L, Shah M. Risk taking behavior in the wake of natural disasters. Journal of Human Resources. 2015; 50 :484–515. [ Google Scholar ]
• Campos-Mercade P, Meier AN, Schneider FH, Wengstrom E. Prosociality predicts health behaviors during the Covid-19 pandemic. Journal of Public Economics. 2021; 195 :104367. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Cardak B, Ip E, Salamanca N, Vecci J. Stability of risk preferences: New evidence from developing and developed countries. Mimeo, University of Gothenburg; 2021. [ Google Scholar ]
• Cardenas JC. How do groups solve local commons dilemmas? Lessons from experimental economics in the field. Environment, Development and Sustainability. 2000; 2 :305–322. [ Google Scholar ]
• Cardenas JC, Carpenter J. Behavioural development economics: lessons from field labs in the developing world. Journal of Development Studies. 2008; 44 :311–338. [ Google Scholar ]
• Cardenas JC, Ostrom E. What do people bring into the game? Experiments in the field about cooperation in the commons. Agricultural Systems. 2004; 82 :307–326. [ Google Scholar ]
• Cassar A, Healey A, Von Kessler C. Trust, risk, and time preferences after a natural disaster: experimental evidence from Thailand. World Development. 2017; 94 :90–105. [ Google Scholar ]
• Castillo M, Petrie R, Torero M. On the preferences of principals and agents. Economic Inquiry. 2010; 48 :266–273. [ Google Scholar ]
• Cecchi F, Duchoslav J. The effect of prenatal stress on cooperation: evidence from violent conflict in Uganda. European Economic Review. 2018; 101 :35–56. [ Google Scholar ]
• Charness G, Gneezy U, Kuhn M. Experimental methods: extra-laboratory experiments extending the reach of experimental economics. Journal of Economic Behaviour & Organization. 2013; 91 :93–100. [ Google Scholar ]
• Chattopadhyay R, Duflo E. The impact of reservation in the panchayati raj: evidence from a nationwide randomized experiment. Economic and Political Weekly. 2004; 39 :979–986. [ Google Scholar ]
• Chattopadhyay R, Duflo E. Women as policy makers: evidence from a randomized policy experiment in India. Econometrica. 2004; 72 :1409–1443. [ Google Scholar ]
• Chaudhuri, A., Iversen, V., Jensenius, F. R., & Maitra, P. (2020). Time in office and the changing gender gap in dishonesty . Working Paper 8217. Munich, Germany: CESifo.
• Chaudhuri A, Iversen V, Jensenius FR, Maitra P. Selecting the best of Us? Politician quality in village councils in West Bengal, India. University of Auckland; 2021. [ Google Scholar ]
• Chen Y, Li SX. Group identity and social preferences. American Economic Review. 2009; 99 :33–50. [ Google Scholar ]
• Cohn A, Maréchal M, Noll T. Bad boys: How criminal identity salience affects rule violation. Review of Economic Studies. 2015; 82 :1289–1308. [ Google Scholar ]
• Cohn A, Maréchal M. Laboratory measure of cheating predicts school misconduct. Economic Journal. 2019; 128 :2743–2754. [ Google Scholar ]
• Cohn A, Marechal M, Fehr E. Business culture and dishonesty in the banking industry. Nature. 2014; 516 :86–89. [ PubMed ] [ Google Scholar ]
• Cooper DJ, Kagel JH, Lo W, Gu QL. Gaming against managers in incentive systems: Experimental results with Chinese students and Chinese managers. American Economic Review. 1999; 89 :781–804. [ Google Scholar ]
• Cubitt RP, Read D. Can intertemporal choice experiments elicit time preferences for consumption? Experimental Economics. 2007; 10 :369–389. [ Google Scholar ]
• Dasgupta U, Gangadharan L, Maitra P, Mani S, Subramanian S. Choosing to be trained: do behavioral traits matter? Journal of Economic Behavior and Organization. 2015; 110 :145–159. [ Google Scholar ]
• Dasgupta U, Mani S. Only mine or all ours: do stronger entitlements affect altruistic choices in the household. World Development. 2015; 67 :363–375. [ Google Scholar ]
• De Paola M, Scoppa V, Lombardo R. Can gender quotas break down negative stereotypes? Evidence from changes in electoral rules. Journal of Public Economics. 2010; 94 :344–353. [ Google Scholar ]
• de Quidt J, Haushofer J, Roth C. Measuring and bounding experimenter demand. American Economic Review. 2018; 108 :3266–3301. [ Google Scholar ]
• Dercon S, Christiaensen L. Consumption risk, technology adoption and poverty traps: evidence from Ethiopia. Journal of Development Economics. 2011; 96 :159–173. [ Google Scholar ]
• Derry P, Baum A. The role of the experimenter in field studies of distressed populations. Journal of Traumatic Stress. 1994; 7 :625–635. [ PubMed ] [ Google Scholar ]
• Dhar D, Jain T, Jayachandran S. Reshaping adolescents’ gender attitudes: Evidence from a school-based experiment in India. IIM (A) and Northwestern University; 2020. [ Google Scholar ]
• Dimant E, van Kleef GA, Shalvi S. Requiem for a nudge: Framing effects in nudging honesty. Journal of Economic Behavior & Organization. 2020; 172 :247–266. [ Google Scholar ]
• Duflo E. Women empowerment and economic development. Journal of Economic Literature. 2012; 50 :1051–1079. [ Google Scholar ]
• Eckbo, B. E., Nygaard, K., & Thorburn, K. S. (2019). Board gender-balancing and firm value . European Corporate Governance Institute (ECGI)—Finance Working Paper No. 463/2016; Tuck School of Business Working Paper No. 2746786.
• Eckel C, Gangadharan L, Grossman PJ, Xue N. The gender leadership gap: Insights from experiments. In: Chaudhuri A, editor. Research agenda in experimental economics. Cheltenham, UK: Edward Elgar; 2021. [ Google Scholar ]
• Eckel C, El-Gamal MA, Wilson R. Risk loving after the storm: a bayesian-network study of hurricane katrina evacuees. Journal of Economic Behavior and Organization. 2009; 69 :110–124. [ Google Scholar ]
• Eckel CC, Grossman PJ. Men, women and risk aversion: experimental evidence. In: Plott CR, Smith VL, editors. Handbook of experimental economics results. North-Holland; 2008. pp. 1061–1073. [ Google Scholar ]
• Edlund L, Pande R. Why have women become left-wing? The political gender gap and the decline of marriage. Quarterly Journal of Economics. 2001; 117 :917–961. [ Google Scholar ]
• Erkal N, Gangadharan L, Koh BH. Monetary and non-monetary incentives in real-effort tournaments. European Economic Review. 2018; 101 :528–545. [ Google Scholar ]
• Exley CL, Niederle M, Vesterlund L. Knowing when to ask: the cost of leaning in. Journal of Political Economy. 2020; 128 :816–854. [ Google Scholar ]
• Falk A, Heckman JJ. Lab experiments are a major source of knowledge in the social sciences. Science. 2009; 23 :535–538. [ PubMed ] [ Google Scholar ]
• Fehr E, Fischbacher U. Third-party punishment and social norms. Economics and Human Behavior. 2004; 25 :63–87. [ Google Scholar ]
• Fehr E, List JA. The hidden costs and returns of incentives—trust and trustworthiness among Ceos. Journal of the European Economic Association. 2010; 2 :743–771. [ Google Scholar ]
• Fischbacher U, Föllmi-Heusi F. Lies in disguise—an experimental study on cheating. Journal of the European Economic Association. 2013; 11 :525–547. [ Google Scholar ]
• Frechette, G. R. (2011). Laboratory Experiments: Professionals Versus Students, NYU.
• Friedman D, Sunder S. Experimental methods: a primer for economists. Cambridge University Press; 1994. [ Google Scholar ]
• Gächter S, Nosenzo D, Sefton M. Peer effects in pro-social behavior: social norms or social preferences? Journal of the European Economic Association. 2013; 11 :548–573. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Gächter S, Schulz JF. Intrinsic honesty and the prevalence of rule violations across societies. Nature. 2016; 531 :496–499. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Gangadharan L, Islam A, Ouch C, Wang LC. The long-term effects of genocide on antisocial preferences. Monash University; 2020. [ Google Scholar ]
• Gangadharan L, Jain T, Maitra P. Third party response to transgression. Monash University and IIM(A); 2021. [ Google Scholar ]
• Gangadharan L, Jain T, Maitra P, Vecci J. Social identity and governance: the behavioral response to female leaders. European Economic Review. 2016; 90 :302–325. [ Google Scholar ]
• Gangadharan L, Jain T, Maitra P, Vecci J. Female leaders and their response to the social environment. Journal of Economic Behavior and Organization. 2019; 164 :256–272. [ Google Scholar ]
• Gelinas L, Largent EA, Cohen IG, Kornetsky S, Bierer BE, Lynch HF. A framework for ethical payment to research participants. New England Journal of Medicine. 2018; 378 :766–771. [ PubMed ] [ Google Scholar ]
• Gneezy U, Potters J. An experiment on risk taking and evaluation periods. The Quarterly Journal of Economics. 1997; 112 :631–645. [ Google Scholar ]
• Gneezy U, Leonard KL, List J. Gender differences in competition: evidence from a matrilineal and patriarchal society. Econometrica. 2009; 77 :1637–1664. [ Google Scholar ]
• Gneezy U, Rustichini A. A fine is a price. The Journal of Legal Studies. 2000; 29 :1–17. [ Google Scholar ]
• Golsteyn BHH, Grönqvist H, Lindahl L. Adolescent time preferences predict lifetime outcomes. Economic Journal. 2014; 124 :F739–F761. [ Google Scholar ]
• Gong B, Yan H, Yang C-L. Gender differences in the dictator experiment: evidence from the matrilineal Mosuo and the patriarchal Yi. Experimental Economics. 2015; 18 :302–313. [ Google Scholar ]
• Grady C. The continued complexities of paying research participants. American Journal of Bioethics. 2019; 19 :5–7. [ PubMed ] [ Google Scholar ]
• Greif A. Contract enforceability and economic institutions in early trade: the magjribi traders’ coalition. American Economic Review. 1993; 83 :525–548. [ Google Scholar ]
• Greif A. Cultural beliefs and the organization of society: A historical and theoretical reflection on collectivist and individualist societies. Journal of Political Economy. 1994; 102 :912–950. [ Google Scholar ]
• Guo S, Liang P, Xiao E. In-group bias in prison. Games and Economic Behavior. 2020; 122 :328–340. [ Google Scholar ]
• Gupta G, Mahmud M, Maitra P, Mitra S, Neelim A. Religion, minority status and trust: Evidence from a field experiment. Journal of Economic Behavior & Organization. 2018; 146 :180–205. [ Google Scholar ]
• Hanaoka C, Shigeoka H, Watanabe Y. Do risk preferences change? Evidence from the great east japan earthquake. Amerian Economic Journal: Applied Economics. 2018; 10 :298–330. [ Google Scholar ]
• Hanna R, Wang SY. Dishonesty and selection into public service: evidence from India. American Economic Journal: Economic Policy. 2017; 9 :262–290. [ Google Scholar ]
• Harbaugh W, Krause K. Children’s altruism in public good and dictator experiments. Economic Inquiry. 2000; 38 :95–109. [ Google Scholar ]
• Harrison G, List JA. Field experiments. Journal of Economic Literature. 2004; 42 :1009–1055. [ Google Scholar ]
• Heinrich J, Heine SJ, Norenzayan A. Beyond weird: towards a broad-based behavioral science. Behavioral and Brain Sciences. 2010; 33 :111. [ Google Scholar ]
• Henrich J. Does culture matter in economic behavior? Ultimatum game bargaining among the Machiguenga of the Peruvian Amazon. American Economic Review. 2000; 90 :973–979. [ Google Scholar ]
• Henrich J, Boyd R, Bowles S, Camerer C, Fehr E, Gintis H, McElreath R. In Search of homo economicus: behavioral experiments in 15 small-scale societies. American Economic Review: Papers and Proceedings. 2001; 91 :73–78. [ Google Scholar ]
• Henrich J, McElreath R, Barr A, Ensminger J, Barrett C, Bolyanatz A, Cardenas J, Gurven M, Gwako E, Henrich N, Lesorogol C, Marlowe F, Tracer D, Ziker J. Costly punishment across human societies. Science. 2006; 312 :1767–1770. [ PubMed ] [ Google Scholar ]
• Holt C, Laury SK. Risk aversion and incentive effects. American Economic Review. 2002; 92 :1644–1655. [ Google Scholar ]
• Islam, A., Leister, C. M., Mahmud, M., & Raschky, P. (2020). Natural disaster and risk-sharing behavior: evidence from rural Bangladesh. Journal of Risk and Uncertainty (forthcoming).
• Iversen V, Jackson C, Kebede B, Munro A, Verschoor A. Do spouses realise cooperative gains? Experimental evidence from rural Uganda. World Development. 2011; 30 :569–578. [ Google Scholar ]
• Jayachandran, S. (2020). Social norms as a barrier to women’s employment in developing countries . NBER Working Paper 27449.
• Krogstad D, Diop S, Diallo A, Mzayek F, Keating J, Koita O, Toure Y. Informed consent in international research: the rationale for different approaches. American Journal of Tropical Medicine and Hygiene. 2010; 83 :743–747. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Kröll M, Rustagi D. Measuring honesty and explaining adulteration in naturally occurring markets. University of Nottingham; 2020. [ Google Scholar ]
• Krupka EL, Weber RA. Identifying Social norms using coordination games: Why does dictator game sharing vary? Journal of the European Economic Association. 2013; 11 :495–524. [ Google Scholar ]
• Lassebie J. Gender quotas and the selection of local politicians: Evidence from french municipal elections. European Journal of Political Economy. 2020; 62 :101842. [ Google Scholar ]
• Leibbrandt A, Maitra P, Neelim A. Large stakes and little honesty? Experimental evidence from a developing country. Economics Letters. 2018; 169 :76–79. [ Google Scholar ]
• List J, Rasul I. Field experiments in labor economics. In: Ashenfelter O, Card D, editors. Handbook of labor economics volume 4a. Elsevier; 2011. pp. 104–228. [ Google Scholar ]
• List J, Mason CF. Are ceos expected utility maximizers? Journal of Econometrics. 2011; 162 :114–123. [ Google Scholar ]
• List JA, Price MK. Using field experiments in environmental and resource economics. Review of Environmental Economics and Policy. 2016; 10 :206–225. [ Google Scholar ]
• List JA, Reiley DH. Field Experiments in Economics: Palgrave Entry, in The New Palgrave Dictionary of Economics . Second Edition. 2008 doi: 10.1111/J.0042-7092.2007.00700.X. [ CrossRef ] [ Google Scholar ]
• Liu E. Time to change what to sow: risk preferences and technology adoption decisions of cotton farmers in China. Review of Economics and Statistics. 2013; 95 :1386–1403. [ Google Scholar ]
• Lott JR, Kenny LW. Did women's sufferage change the size and scope of government? Journal of Political Economy. 1999; 107 :1163–1198. [ Google Scholar ]
• Ludwig J, Kling J, Mullainathan S. Mechanism experiments and policy evaluations. Journal of Economic Perspectives. 2011; 25 :17–38. [ Google Scholar ]
• Maitra P, Neelim A. Behavioral characteristics, stability of preferences and entrepreneurial success. In: Chaudhuri A, editor. Research agenda in experimental economics. Cheltenham, UK: Edward Elgar; 2021. [ Google Scholar ]
• Maitra P, Mani S. Learning and earning: evidence from a randomized evaluation in India. Labour Economics. 2017; 45 :116–130. [ Google Scholar ]
• Masekesa F, Munro A. Intra-household inequality, fairness and productivity. Evidence from a real effort experiment. World Development. 2020; 127 :104763. [ Google Scholar ]
• Matsa DA, Miller AR. A female style in corporate leadership? evidence from quotas. American Economic Journal. 2013; 5 :136–169. [ Google Scholar ]
• Mukherjee S. Gender, group and moral hazard in microfinance: Evidence from matrlineal and patrilineal societies in India. Mumbai: Mimeo, Meghnad Desai Academy; 2018. [ Google Scholar ]
• Newell RG, Siikamäki J. Individual time preferences and energy efficiency. American Economic Review. 2015; 105 :196–200. [ Google Scholar ]
• O'Brien D, Rickne J. Gender quotas and women's political leadership. American Political Science Review. 2016; 110 :112–126. [ Google Scholar ]
• Olsen AL, Hjorth F, Harmon N, Barfort S. Behavioral dishonesty in the public sector. Journal of Public Administration Research and Theory. 2019; 29 :572–590. [ Google Scholar ]
• Page L, Savage D, Torgler B. Variation in risk seeking behaviour following large losses: a natural experiment. European Economic Review. 2014; 71 :121–131. [ Google Scholar ]
• Pande, R., & Ford, D. (2012). Gender quotas and female leadership, world development report: Gender equality and development .
• Plott CR. Industrial organization theory and experimental economics. Journal of Economic Literature. 1982; 20 :1485–1527. [ Google Scholar ]
• Prediger S, Vollan B, Herrmann B. Resource scarcity and antisocial behavior. Journal of Public Economics. 2014; 119 :1–9. [ Google Scholar ]
• Price MK. Using field experiments to address environmental externalities and resource scarcity: major lessons learned and new directions for future research. Oxford Review of Economic Policy. 2014; 30 :621–638. [ Google Scholar ]
• Resnick D. Bioethical Issues in providing financial incentives to research participants. Medicolegal and Bioethics. 2015; 5 :35–41. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Reuben E, Winden F. Social ties and coordination on negative reciprocity: the role of affect. Journal of Public Economics. 2008; 92 :34–53. [ Google Scholar ]
• Roth A, Prasnikar V, Okuno-Fujiwara M, Zamir S. Bargaining and market behavior in Jerusalem, Ljubljana, Pittsburgh, and Tokyo: an experimental study. American Economic Review. 1991; 81 :1968–1095. [ Google Scholar ]
• Samek, A. (2019). Advantages and Disadvantages of Field Experiments, in Handbook of Research Methods and Applications in Experimental Economics , ed. by A. Schram, and A. Ule.: Edward Elgar Publishing
• Sarsons H, Gërxhani K, Reuben E, Schram A. Gender differences in recognition for group work. Journal of Political Economy. 2021; 129 :101–147. [ Google Scholar ]
• Schlag KH, Tremewan J, van der Weele JJ. A penny for your thoughts: A survey of methods for eliciting beliefs. Experimental Economics. 2015; 18 :457–490. [ Google Scholar ]
• Schotter A, Trevino I. Belief elicitation in the laboratory. Annual Review of Economics. 2014; 6 :103–128. [ Google Scholar ]
• Smith VL. Experimental economics: induced value theory. American Economic Review. 1976; 66 :274–279. [ Google Scholar ]
• Smith VL. Theory, experiment and economics. Journal of Economic Perspectives. 1989; 3 :151–169. [ Google Scholar ]
• Smith VL. Economics in the laboratory. Journal of Economic Perspectives. 1994; 8 :113–131. [ Google Scholar ]
• Snowberg E, Yariv L. Testing the waters: behavior across participant pools. American Economic Review. 2021; 111 :687–719. [ Google Scholar ]
• Sutter M, Kocher M, Rützler D, Trautmann S. Impatience and uncertainty: experimental decisions predict adolescents’ field behavior. American Economic Review. 2013; 103 :510–531. [ Google Scholar ]
• Tanaka T, Camerer CF, Nguyen Q. Risk and time preferences: linking experimental and household survey data from vietnam. American Economic Review. 2010; 100 :557–571. [ Google Scholar ]
• Viceisza A. Creating a lab in the field: Economics experiments for policy making. Journal of Economic Surveys. 2015; 30 :835–854. [ Google Scholar ]
• Xiao E, Houser D. Emotion expression in human punishment behavior. Proceedings of the National Academy of Sciences of the United States of America. 2005; 102 :7398–7401. [ PMC free article ] [ PubMed ] [ Google Scholar ]

Learning Materials

• Business Studies
• Combined Science
• Computer Science
• Engineering
• English Literature
• Environmental Science
• Human Geography
• Macroeconomics
• Microeconomics
• Field Experiment

Sometimes, a laboratory setting isn't the best option for investigating a phenomenon when conducting research. Whilst lab experiments offer a lot of control, they are artificial and do not truly represent the real world, which causes issues with ecological validity. This is where field experiments come in. 

Millions of flashcards designed to help you ace your studies

• Cell Biology

Which of the following statement is correct? 

What type of validity is mundane realism related to? 

What are the potential ethical issues that may happen when doing a field experiment? 

True or false: A potentially biased sample is an issue for natural experiments. 

True or false: In field experiments, variables are manipulated. 

Can variables be manipulated in real-life settings? 

A study aimed to explore how early parental interactions influenced later attachment styles. Would it be appropriate to use a field experiment? 

Researchers have a higher level of control in           experiments compared to               experiments.

A study aimed to explore how children responded to authoritative figures versus non-authoritative ones at school. Could a field experiment be used? 

Can a field experiment be used to compare children's behaviour around their usual and substitute teachers? 

Review generated flashcards

to start learning or create your own AI flashcards

Start learning or create your own AI flashcards

StudySmarter Editorial Team

Team Field Experiment Teachers

• 5 minutes reading time
• Checked by StudySmarter Editorial Team
• Approaches in Psychology
• Basic Psychology
• Biological Bases of Behavior
• Biopsychology
• Careers in Psychology
• Clinical Psychology
• Cognition and Development
• Cognitive Psychology
• Data Handling and Analysis
• Developmental Psychology
• Eating Behaviour
• Emotion and Motivation
• Famous Psychologists
• Forensic Psychology
• Health Psychology
• Individual Differences Psychology
• Issues and Debates in Psychology
• Personality in Psychology
• Psychological Treatment
• Relationships
• Research Methods in Psychology
• Aims and Hypotheses
• Causation in Psychology
• Coding Frame Psychology
• Correlational Studies
• Cross Cultural Research
• Cross Sectional Research
• Ethical Issues and Ways of Dealing with Them
• Experimental Designs
• Features of Science
• Independent Group Design
• Lab Experiment
• Longitudinal Research
• Matched Pairs Design
• Meta Analysis
• Natural Experiment
• Observational Design
• Online Research
• Paradigms and Falsifiability
• Peer Review and Economic Applications of Research
• Pilot Studies and the Aims of Piloting
• Quality Criteria
• Questionnaire Construction
• Repeated Measures Design
• Research Methods
• Sampling Frames
• Sampling Psychology
• Scientific Processes
• Scientific Report
• Scientific Research
• Self-Report Design
• Self-Report Techniques
• Semantic Differential Rating Scale
• Snowball Sampling
• Schizophrenia
• Scientific Foundations of Psychology
• Scientific Investigation
• Sensation and Perception
• Social Context of Behaviour
• Social Psychology

Jump to a key chapter

Despite its name, field experiments, whilst they can be conducted in a field, are not restricted to a literal field.

Both laboratory and field experiments manipulate a variable to see if it can be controlled and affect the dependent variable. Also, both are valid forms of experimentation.

• We will start by learning the field experiment definition and identify how field experiments are used in research.
• Moving on from this, we will explore a field experiment example carried out by Hofling in 1966.
• Finally, we will discuss the field experiment's advantages and disadvantages.

Field Experiment Definition

A field experiment is a research method where the independent variable is manipulated, and the dependent variable is measured in a real-world setting.

If you had to research travel, a field experiment could be performed on a train. Also, you could analyse a car or bike ride out in the streets. Similarly, someone might conduct an experiment in a school investigating different phenomena present in classrooms or school playgrounds.

Field Experiment: Psychology

Field experiments are usually designed and used in psychology when researchers want to observe participants in their natural environment, but the phenomenon is not naturally occurring. Therefore, the researcher must manipulate the investigated variables to measure the outcome, e.g. how students behaviour when a teacher or a substitute teacher is present.

The procedure of field experiments in psychology is the following:

• Identify a research question, variables , and hypotheses.
• Recruit participants.
• Carry out the investigation.
• Analyse data and report results.

Field Experiment: Example

Hofling (1966) conducted a field experiment to investigate obedience in nurses. The study recruited 22 nurses working in a psychiatric hospital on a night shift, although they were unaware they were taking part in the study.

D uring their shift, a doctor, who was actually the researcher, called the nurses and asked them to urgently administer 20mg of a drug to a patient (double the maximum dosage). The doctor/researcher told the nurses that he would authorise medication administration later.

The research aimed to identify if people broke the rules and obeyed authoritative figures' orders.

The results showed that 95% of the nurses obeyed the order, despite breaking the rules. Only one questioned the doctor.

The Hofling study is an example of a field experiment. It was carried out in a natural setting, and the researcher manipulated the situation (instructed nurses to administer high-dosage medication) to see if it affected whether nurses obeyed the authoritative figure or not.

Field Experiment: Advantages and Disadvantages

Like any type of research, field experiments have certain advantages and disadvantages that must be considered before opting for this research method.

Field Experiments: Advantages

Some of the advantages of field experiments include the following:

• The results are more likely to reflect real-life compared to laboratory research, as they have higher ecological validity.

Hawthorne effect is when people adjust their behaviour because they know they are being observed.

• It is high in mundane realism compared to lab research; this refers to the extent to which the setting and materials used in a study reflect real-life situations. Field experiments have high mundane realism. Thus, they have high external validity.

A field experiment would be an appropriate research design when investigating children's behaviour changes at school. More specifically, to compare their behaviours around their usual and substitute teachers.

• It can establish c ausal relationships because researchers manipulate a variable and measure its effect. However, extraneous variables can make this difficult. We will address these issues in the next paragraph.

Field Experiments: Disadvantages

The disadvantages of field experiments are the following:

• Researchers have less control over extraneous/confounding variables, reducing confidence in establishing causal relationships .
• It is difficult to replicate the research, making it hard to determine the results' reliability.
• This experimental method has a high chance of collecting a biased sample, making it difficult to generalise the results.
• It may not be easy to record data accurately with so many variables present. Overall, field experiments have less control.
• Potential ethical issues of field experiments include: difficulty getting informed consent, and the researcher may need to deceive participants.

Field Experiment - Key Takeaways

• The field experiment definition is a research method where the independent variable is manipulated, and the dependent variable is measured in a real-world setting.
• Field experiments are usually used in psychology when researchers want to observe participants in their natural environment. The phenomenon is not naturally occurring, so the researcher must manipulate the variables to measure the outcome.
• Hofling (1966) used a field experiment to investigate if nurses wrongfully obeyed authoritative figures in their workplace.
• Field experiments have high ecological validity, establish causal relationships, and reduce the chances of demand characteristics interfering with research.
• However, they offer less control, and confounding variables may be an issue. From the ethical perspective, participants cannot always consent to participate and may need to be deceived to be observed. Replicating field experiments is also difficult.

Flashcards in Field Experiment 10

The researcher manipulates the variables in a controlled setting.

External validity.

Deception of participants.

Learn with 10 Field Experiment flashcards in the free StudySmarter app

We have 14,000 flashcards about Dynamic Landscapes.

Already have an account? Log in

Frequently Asked Questions about Field Experiment

What is a field experiment? 

A field experiment is a research method where the independent variable is manipulated, and the dependent variable is measured in a real-world setting. 

What is the difference between natural and field experiments?

In field experiments, researchers manipulate the independent variable. On the other hand, in natural experiments, the researcher does not manipulate anything in the investigation. 

What is an example of a field experiment? 

Hofling (1966) utilised a field experiment to identify if nurses would break the rules and obey an authoritative figure. 

What is one drawback of field experiments? 

A disadvantage of a field experiment is that researchers cannot control the extraneous variables, and this may reduce the validity of the findings. 

How to conduct a field experiment?

The steps for conducting a field experiment are: 

• identify a research question, variables, and hypotheses 
• recruit participants
• carry out the experiment
• analyse the data and report the results 

Test your knowledge with multiple choice flashcards

Join the StudySmarter App and learn efficiently with millions of flashcards and more!

Keep learning, you are doing great.

Discover learning materials with the free StudySmarter app

About StudySmarter

StudySmarter is a globally recognized educational technology company, offering a holistic learning platform designed for students of all ages and educational levels. Our platform provides learning support for a wide range of subjects, including STEM, Social Sciences, and Languages and also helps students to successfully master various tests and exams worldwide, such as GCSE, A Level, SAT, ACT, Abitur, and more. We offer an extensive library of learning materials, including interactive flashcards, comprehensive textbook solutions, and detailed explanations. The cutting-edge technology and tools we provide help students create their own learning materials. StudySmarter’s content is not only expert-verified but also regularly updated to ensure accuracy and relevance.

Team Psychology Teachers

Study anywhere. Anytime.Across all devices.

Create a free account to save this explanation..

Save explanations to your personalised space and access them anytime, anywhere!

By signing up, you agree to the Terms and Conditions and the Privacy Policy of StudySmarter.

Sign up to highlight and take notes. It’s 100% free.

Join over 22 million students in learning with our StudySmarter App

The first learning app that truly has everything you need to ace your exams in one place

• Flashcards & Quizzes
• AI Study Assistant
• Study Planner
• Smart Note-Taking