random sampling is to blank as random assignment is to

Random Assignment in Psychology: Definition & Examples

Julia Simkus

Editor at Simply Psychology

BA (Hons) Psychology, Princeton University

Julia Simkus is a graduate of Princeton University with a Bachelor of Arts in Psychology. She is currently studying for a Master's Degree in Counseling for Mental Health and Wellness in September 2023. Julia's research has been published in peer reviewed journals.

Learn about our Editorial Process

Saul Mcleod, PhD

Editor-in-Chief for Simply Psychology

BSc (Hons) Psychology, MRes, PhD, University of Manchester

Saul Mcleod, PhD., is a qualified psychology teacher with over 18 years of experience in further and higher education. He has been published in peer-reviewed journals, including the Journal of Clinical Psychology.

Olivia Guy-Evans, MSc

Associate Editor for Simply Psychology

BSc (Hons) Psychology, MSc Psychology of Education

Olivia Guy-Evans is a writer and associate editor for Simply Psychology. She has previously worked in healthcare and educational sectors.

In psychology, random assignment refers to the practice of allocating participants to different experimental groups in a study in a completely unbiased way, ensuring each participant has an equal chance of being assigned to any group.

In experimental research, random assignment, or random placement, organizes participants from your sample into different groups using randomization. 

Random assignment uses chance procedures to ensure that each participant has an equal opportunity of being assigned to either a control or experimental group.

The control group does not receive the treatment in question, whereas the experimental group does receive the treatment.

When using random assignment, neither the researcher nor the participant can choose the group to which the participant is assigned. This ensures that any differences between and within the groups are not systematic at the onset of the study. 

In a study to test the success of a weight-loss program, investigators randomly assigned a pool of participants to one of two groups.

Group A participants participated in the weight-loss program for 10 weeks and took a class where they learned about the benefits of healthy eating and exercise.

Group B participants read a 200-page book that explains the benefits of weight loss. The investigator randomly assigned participants to one of the two groups.

The researchers found that those who participated in the program and took the class were more likely to lose weight than those in the other group that received only the book.

Importance 

Random assignment ensures that each group in the experiment is identical before applying the independent variable.

In experiments , researchers will manipulate an independent variable to assess its effect on a dependent variable, while controlling for other variables. Random assignment increases the likelihood that the treatment groups are the same at the onset of a study.

Thus, any changes that result from the independent variable can be assumed to be a result of the treatment of interest. This is particularly important for eliminating sources of bias and strengthening the internal validity of an experiment.

Random assignment is the best method for inferring a causal relationship between a treatment and an outcome.

Random Selection vs. Random Assignment 

Random selection (also called probability sampling or random sampling) is a way of randomly selecting members of a population to be included in your study.

On the other hand, random assignment is a way of sorting the sample participants into control and treatment groups. 

Random selection ensures that everyone in the population has an equal chance of being selected for the study. Once the pool of participants has been chosen, experimenters use random assignment to assign participants into groups. 

Random assignment is only used in between-subjects experimental designs, while random selection can be used in a variety of study designs.

Random Assignment vs Random Sampling

Random sampling refers to selecting participants from a population so that each individual has an equal chance of being chosen. This method enhances the representativeness of the sample.

Random assignment, on the other hand, is used in experimental designs once participants are selected. It involves allocating these participants to different experimental groups or conditions randomly.

This helps ensure that any differences in results across groups are due to manipulating the independent variable, not preexisting differences among participants.

When to Use Random Assignment

Random assignment is used in experiments with a between-groups or independent measures design.

In these research designs, researchers will manipulate an independent variable to assess its effect on a dependent variable, while controlling for other variables.

There is usually a control group and one or more experimental groups. Random assignment helps ensure that the groups are comparable at the onset of the study.

How to Use Random Assignment

There are a variety of ways to assign participants into study groups randomly. Here are a handful of popular methods: 

• Random Number Generator : Give each member of the sample a unique number; use a computer program to randomly generate a number from the list for each group.
• Lottery : Give each member of the sample a unique number. Place all numbers in a hat or bucket and draw numbers at random for each group.
• Flipping a Coin : Flip a coin for each participant to decide if they will be in the control group or experimental group (this method can only be used when you have just two groups) 
• Roll a Die : For each number on the list, roll a dice to decide which of the groups they will be in. For example, assume that rolling 1, 2, or 3 places them in a control group and rolling 3, 4, 5 lands them in an experimental group.

When is Random Assignment not used?

• When it is not ethically permissible: Randomization is only ethical if the researcher has no evidence that one treatment is superior to the other or that one treatment might have harmful side effects. 
• When answering non-causal questions : If the researcher is just interested in predicting the probability of an event, the causal relationship between the variables is not important and observational designs would be more suitable than random assignment. 
• When studying the effect of variables that cannot be manipulated: Some risk factors cannot be manipulated and so it would not make any sense to study them in a randomized trial. For example, we cannot randomly assign participants into categories based on age, gender, or genetic factors.

Drawbacks of Random Assignment

While randomization assures an unbiased assignment of participants to groups, it does not guarantee the equality of these groups. There could still be extraneous variables that differ between groups or group differences that arise from chance. Additionally, there is still an element of luck with random assignments.

Thus, researchers can not produce perfectly equal groups for each specific study. Differences between the treatment group and control group might still exist, and the results of a randomized trial may sometimes be wrong, but this is absolutely okay.

Scientific evidence is a long and continuous process, and the groups will tend to be equal in the long run when data is aggregated in a meta-analysis.

Additionally, external validity (i.e., the extent to which the researcher can use the results of the study to generalize to the larger population) is compromised with random assignment.

Random assignment is challenging to implement outside of controlled laboratory conditions and might not represent what would happen in the real world at the population level. 

Random assignment can also be more costly than simple observational studies, where an investigator is just observing events without intervening with the population.

Randomization also can be time-consuming and challenging, especially when participants refuse to receive the assigned treatment or do not adhere to recommendations. 

What is the difference between random sampling and random assignment?

Random sampling refers to randomly selecting a sample of participants from a population. Random assignment refers to randomly assigning participants to treatment groups from the selected sample.

Does random assignment increase internal validity?

Yes, random assignment ensures that there are no systematic differences between the participants in each group, enhancing the study’s internal validity .

Does random assignment reduce sampling error?

Yes, with random assignment, participants have an equal chance of being assigned to either a control group or an experimental group, resulting in a sample that is, in theory, representative of the population.

Random assignment does not completely eliminate sampling error because a sample only approximates the population from which it is drawn. However, random sampling is a way to minimize sampling errors. 

When is random assignment not possible?

Random assignment is not possible when the experimenters cannot control the treatment or independent variable.

For example, if you want to compare how men and women perform on a test, you cannot randomly assign subjects to these groups.

Participants are not randomly assigned to different groups in this study, but instead assigned based on their characteristics.

Does random assignment eliminate confounding variables?

Yes, random assignment eliminates the influence of any confounding variables on the treatment because it distributes them at random among the study groups. Randomization invalidates any relationship between a confounding variable and the treatment.

Why is random assignment of participants to treatment conditions in an experiment used?

Random assignment is used to ensure that all groups are comparable at the start of a study. This allows researchers to conclude that the outcomes of the study can be attributed to the intervention at hand and to rule out alternative explanations for study results.

Further Reading

• Bogomolnaia, A., & Moulin, H. (2001). A new solution to the random assignment problem .  Journal of Economic theory ,  100 (2), 295-328.
• Krause, M. S., & Howard, K. I. (2003). What random assignment does and does not do .  Journal of Clinical Psychology ,  59 (7), 751-766.

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• What Is Probability Sampling? | Types & Examples

What Is Probability Sampling? | Types & Examples

Published on July 5, 2022 by Kassiani Nikolopoulou . Revised on June 22, 2023.

Probability sampling is a sampling method that involves randomly selecting a sample, or a part of the population that you want to research. It is also sometimes called random sampling.

To qualify as being random, each research unit (e.g., person, business, or organization in your population) must have an equal chance of being selected. This is usually done through a random selection process, like a drawing.

Table of contents

Types of probability sampling, examples of probability sampling methods, probability vs. non-probability sampling, advantages and disadvantages of probability sampling, other interesting articles, frequently asked questions about probability sampling.

There are four commonly used types of probability sampling designs:

Simple random sampling

• Stratified sampling

Systematic sampling

• Cluster sampling

Simple random sampling gathers a random selection from the entire population, where each unit has an equal chance of selection. This is the most common way to select a random sample.

To compile a list of the units in your research population, consider using a random number generator. There are several free ones available online, such as random.org , calculator.net , and randomnumbergenerator.org .

Writing down the names of all 4,000 inhabitants by hand to randomly draw 100 of them would be impractical and time-consuming, as well as questionable for ethical reasons. Instead, you decide to use a random number generator to draw a simple random sample.

Stratified sampling collects a random selection of a sample from within certain strata, or subgroups within the population. Each subgroup is separated from the others on the basis of a common characteristic, such as gender, race, or religion. This way, you can ensure that all subgroups of a given population are adequately represented within your sample population.

For example, if you are dividing a student population by college majors, Engineering, Linguistics, and Physical Education students are three different strata within that population.

To split your population into different subgroups, first choose which characteristic you would like to divide them by. Then you can select your sample from each subgroup. You can do this in one of two ways:

• By selecting an equal number of units from each subgroup
• By selecting units from each subgroup equal to their proportion in the total population

If you take a simple random sample, children from urban areas will have a far greater chance of being selected, so the best way of getting a representative sample is to take a stratified sample.

First, you divide the population into your strata: one for children from urban areas and one for children from rural areas. Then, you take a simple random sample from each subgroup. You can use one of two options:

• Select 100 urban and 100 rural, i.e., an equal number of units
• Select 80 urban and 20 rural, which gives you a representative sample of 100 people

Systematic sampling draws a random sample from the target population by selecting units at regular intervals starting from a random point. This method is useful in situations where records of your target population already exist, such as records of an agency’s clients, enrollment lists of university students, or a company’s employment records. Any of these can be used as a sampling frame.

To start your systematic sample, you first need to divide your sampling frame into a number of segments, called intervals. You calculate these by dividing your population size by the desired sample size.

Then, from the first interval, you select one unit using simple random sampling. The selection of the next units from other intervals depends upon the position of the unit selected in the first interval.

Let’s refer back to our example about the political views of the municipality of 4,000 inhabitants. You can also draw a sample of 100 people using systematic sampling. To do so, follow these steps:

• Determine your interval: 4,000 / 100 = 40. This means that you must select 1 inhabitant from every 40 in the record.
• Using simple random sampling (e.g., a random number generator), you select 1 inhabitant.
• Let’s say you select the 11th person on the list. In every subsequent interval, you need to select the 11th person in that interval, until you have a sample of 100 people.

Cluster sampling is the process of dividing the target population into groups, called clusters. A randomly selected subsection of these groups then forms your sample. Cluster sampling is an efficient approach when you want to study large, geographically dispersed populations. It usually involves existing groups that are similar to each other in some way (e.g., classes in a school).

There are two types of cluster sampling:

• Single (or one-stage) cluster sampling, when you divide the entire population into clusters
• Multistage cluster sampling, when you divide the cluster further into more clusters, in order to narrow down the sample size

Clusters are pre-existing groups, so each high school is a cluster, and you assign a number to each one of them. Then, you use simple random sampling to further select clusters. How many clusters you select will depend on the sample size that you need.

Multi-stage sampling is a more complex form of cluster sampling, in which smaller groups are successively selected from larger populations to form the sample population used in your study.

First, you take a simple random sample of departments. Then, again using simple random sampling, you select a number of units. Based on the size of the population (i.e., how many employees work at the company) and your desired sample size, you establish that you need to include 3 units in your sample.

In stratified sampling , you divide your population in groups (strata) that share a common characteristic and then select some members from every group for your sample. In cluster sampling , you use pre-existing groups to divide your population into clusters and then include all members from randomly selected clusters for your sample.

Receive feedback on language, structure, and formatting

Professional editors proofread and edit your paper by focusing on:

• Academic style
• Vague sentences
• Style consistency

See an example

There are a few methods you can use to draw a random sample. Here are a few examples:

• The fishbowl draw
• A random number generator
• The random number function

Fishbowl draw

You are investigating the use of a popular portable e‐reader device among library and information science students and its effects on individual reading practices. You write the names of 25 students on pieces of paper, put them in a jar, and then, without looking, randomly select three students to be interviewed for your research.

All students have equal chances of being selected and no other consideration (such as personal preference) can influence this selection. This method is suitable when your total population is small, so writing the names or numbers of each unit on a piece of paper is feasible.

Random number generator

Suppose you are researching what people think about road safety in a specific residential area. You make a list of all the suburbs and assign a number to each one of them. Then, using an online random number generator, you select four numbers, corresponding to four suburbs, and focus on them.

This works best when you already have a list with the total population and you can easily assign every individual a number.

RAND function in Microsoft Excel

If your data are in a spreadsheet, you can also use the random number function (RAND) in Microsoft Excel to select a random sample.

Suppose you have a list of 4,000 people and you need a sample of 300. By typing in the formula =RAND() and then pressing enter, you can have Excel assign a random number to each name on the list. For this to work, make sure there are no blank rows.

This video explains how to use the RAND function.

Depending on the goals of your research study, there are two sampling methods you can use:

• Probability sampling : Sampling method that ensures that each unit in the study population has an equal chance of being selected
• Non-probability sampling : Sampling method that uses a non-random sample from the population you want to research, based on specific criteria, such as convenience

Probability sampling

In quantitative research , it is important that your sample is representative of your target population. This allows you to make strong statistical inferences based on the collected data. Having a sufficiently large random probability sample is the best guarantee that the sample will be representative and the results are generalizable and free from research biases like selection bias and sampling bias .

Non-probability sampling

Non-probability sampling designs are used in both quantitative and qualitative research when the number of units in the population is either unknown or impossible to individually identify. It is also used when you want to apply the results only to a certain subsection or organization rather than the general public. Non-probability sampling is at higher risk than probability sampling for research biases like sampling bias .

You are unlikely to be able to compile a list of every practicing organizational psychologist in the country, but you could compile a list with all the experts in your area and select a few to interview.

It’s important to be aware of the advantages and disadvantages of probability sampling, as it will help you decide if this is the right sampling method for your research design.

Advantages of probability sampling

There are two main advantages to probability sampling.

• Samples selected with this method are representative of the population at large. Due to this, inferences drawn from such samples can be generalized to the total population you are studying.
• As some statistical tests, such as multiple linear regression , t test , or ANOVA , can only be applied to a sample size large enough to approximate the true distribution of the population, using probability sampling allows you to establish correlation or cause-and-effect relationship between your variables.

Disadvantages of probability sampling

Choosing probability sampling as your sampling method comes with some challenges, too. These include the following:

• It may be difficult to access a list of the entire population, due to ethical or privacy concerns, or a full list may not exist. It can be expensive and time-consuming to compile this yourself.
• Although probability sampling reduces the risk of sampling bias , it can still occur. When your selected sample is not inclusive enough, representation of the full population is skewed .

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

• Student’s  t -distribution
• Normal distribution
• Null and Alternative Hypotheses
• Chi square tests
• Confidence interval
• Quartiles & Quantiles
• Data cleansing
• Reproducibility vs Replicability
• Peer review
• Prospective cohort study

Research bias

• Implicit bias
• Cognitive bias
• Placebo effect
• Hawthorne effect
• Hindsight bias
• Affect heuristic
• Social desirability bias

When your population is large in size, geographically dispersed, or difficult to contact, it’s necessary to use a sampling method .

This allows you to gather information from a smaller part of the population (i.e., the sample) and make accurate statements by using statistical analysis. A few sampling methods include simple random sampling , convenience sampling , and snowball sampling .

Stratified and cluster sampling may look similar, but bear in mind that groups created in cluster sampling are heterogeneous , so the individual characteristics in the cluster vary. In contrast, groups created in stratified sampling are homogeneous , as units share characteristics.

Relatedly, in cluster sampling you randomly select entire groups and include all units of each group in your sample. However, in stratified sampling, you select some units of all groups and include them in your sample. In this way, both methods can ensure that your sample is representative of the target population .

A sampling frame is a list of every member in the entire population . It is important that the sampling frame is as complete as possible, so that your sample accurately reflects your population.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Nikolopoulou, K. (2023, June 22). What Is Probability Sampling? | Types & Examples. Scribbr. Retrieved April 11, 2024, from https://www.scribbr.com/methodology/probability-sampling/

Is this article helpful?

Kassiani Nikolopoulou

Other students also liked, population vs. sample | definitions, differences & examples, sampling methods | types, techniques & examples, what is a research design | types, guide & examples, "i thought ai proofreading was useless but..".

I've been using Scribbr for years now and I know it's a service that won't disappoint. It does a good job spotting mistakes”

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

6.2 Experimental Design

Learning objectives.

• Explain the difference between between-subjects and within-subjects experiments, list some of the pros and cons of each approach, and decide which approach to use to answer a particular research question.
• Define random assignment, distinguish it from random sampling, explain its purpose in experimental research, and use some simple strategies to implement it.
• Define what a control condition is, explain its purpose in research on treatment effectiveness, and describe some alternative types of control conditions.
• Define several types of carryover effect, give examples of each, and explain how counterbalancing helps to deal with them.

In this section, we look at some different ways to design an experiment. The primary distinction we will make is between approaches in which each participant experiences one level of the independent variable and approaches in which each participant experiences all levels of the independent variable. The former are called between-subjects experiments and the latter are called within-subjects experiments.

Between-Subjects Experiments

In a between-subjects experiment , each participant is tested in only one condition. For example, a researcher with a sample of 100 college students might assign half of them to write about a traumatic event and the other half write about a neutral event. Or a researcher with a sample of 60 people with severe agoraphobia (fear of open spaces) might assign 20 of them to receive each of three different treatments for that disorder. It is essential in a between-subjects experiment that the researcher assign participants to conditions so that the different groups are, on average, highly similar to each other. Those in a trauma condition and a neutral condition, for example, should include a similar proportion of men and women, and they should have similar average intelligence quotients (IQs), similar average levels of motivation, similar average numbers of health problems, and so on. This is a matter of controlling these extraneous participant variables across conditions so that they do not become confounding variables.

Random Assignment

The primary way that researchers accomplish this kind of control of extraneous variables across conditions is called random assignment , which means using a random process to decide which participants are tested in which conditions. Do not confuse random assignment with random sampling. Random sampling is a method for selecting a sample from a population, and it is rarely used in psychological research. Random assignment is a method for assigning participants in a sample to the different conditions, and it is an important element of all experimental research in psychology and other fields too.

In its strictest sense, random assignment should meet two criteria. One is that each participant has an equal chance of being assigned to each condition (e.g., a 50% chance of being assigned to each of two conditions). The second is that each participant is assigned to a condition independently of other participants. Thus one way to assign participants to two conditions would be to flip a coin for each one. If the coin lands heads, the participant is assigned to Condition A, and if it lands tails, the participant is assigned to Condition B. For three conditions, one could use a computer to generate a random integer from 1 to 3 for each participant. If the integer is 1, the participant is assigned to Condition A; if it is 2, the participant is assigned to Condition B; and if it is 3, the participant is assigned to Condition C. In practice, a full sequence of conditions—one for each participant expected to be in the experiment—is usually created ahead of time, and each new participant is assigned to the next condition in the sequence as he or she is tested. When the procedure is computerized, the computer program often handles the random assignment.

One problem with coin flipping and other strict procedures for random assignment is that they are likely to result in unequal sample sizes in the different conditions. Unequal sample sizes are generally not a serious problem, and you should never throw away data you have already collected to achieve equal sample sizes. However, for a fixed number of participants, it is statistically most efficient to divide them into equal-sized groups. It is standard practice, therefore, to use a kind of modified random assignment that keeps the number of participants in each group as similar as possible. One approach is block randomization . In block randomization, all the conditions occur once in the sequence before any of them is repeated. Then they all occur again before any of them is repeated again. Within each of these “blocks,” the conditions occur in a random order. Again, the sequence of conditions is usually generated before any participants are tested, and each new participant is assigned to the next condition in the sequence. Table 6.2 “Block Randomization Sequence for Assigning Nine Participants to Three Conditions” shows such a sequence for assigning nine participants to three conditions. The Research Randomizer website ( http://www.randomizer.org ) will generate block randomization sequences for any number of participants and conditions. Again, when the procedure is computerized, the computer program often handles the block randomization.

Table 6.2 Block Randomization Sequence for Assigning Nine Participants to Three Conditions

Random assignment is not guaranteed to control all extraneous variables across conditions. It is always possible that just by chance, the participants in one condition might turn out to be substantially older, less tired, more motivated, or less depressed on average than the participants in another condition. However, there are some reasons that this is not a major concern. One is that random assignment works better than one might expect, especially for large samples. Another is that the inferential statistics that researchers use to decide whether a difference between groups reflects a difference in the population takes the “fallibility” of random assignment into account. Yet another reason is that even if random assignment does result in a confounding variable and therefore produces misleading results, this is likely to be detected when the experiment is replicated. The upshot is that random assignment to conditions—although not infallible in terms of controlling extraneous variables—is always considered a strength of a research design.

Treatment and Control Conditions

Between-subjects experiments are often used to determine whether a treatment works. In psychological research, a treatment is any intervention meant to change people’s behavior for the better. This includes psychotherapies and medical treatments for psychological disorders but also interventions designed to improve learning, promote conservation, reduce prejudice, and so on. To determine whether a treatment works, participants are randomly assigned to either a treatment condition , in which they receive the treatment, or a control condition , in which they do not receive the treatment. If participants in the treatment condition end up better off than participants in the control condition—for example, they are less depressed, learn faster, conserve more, express less prejudice—then the researcher can conclude that the treatment works. In research on the effectiveness of psychotherapies and medical treatments, this type of experiment is often called a randomized clinical trial .

There are different types of control conditions. In a no-treatment control condition , participants receive no treatment whatsoever. One problem with this approach, however, is the existence of placebo effects. A placebo is a simulated treatment that lacks any active ingredient or element that should make it effective, and a placebo effect is a positive effect of such a treatment. Many folk remedies that seem to work—such as eating chicken soup for a cold or placing soap under the bedsheets to stop nighttime leg cramps—are probably nothing more than placebos. Although placebo effects are not well understood, they are probably driven primarily by people’s expectations that they will improve. Having the expectation to improve can result in reduced stress, anxiety, and depression, which can alter perceptions and even improve immune system functioning (Price, Finniss, & Benedetti, 2008).

Placebo effects are interesting in their own right (see Note 6.28 “The Powerful Placebo” ), but they also pose a serious problem for researchers who want to determine whether a treatment works. Figure 6.2 “Hypothetical Results From a Study Including Treatment, No-Treatment, and Placebo Conditions” shows some hypothetical results in which participants in a treatment condition improved more on average than participants in a no-treatment control condition. If these conditions (the two leftmost bars in Figure 6.2 “Hypothetical Results From a Study Including Treatment, No-Treatment, and Placebo Conditions” ) were the only conditions in this experiment, however, one could not conclude that the treatment worked. It could be instead that participants in the treatment group improved more because they expected to improve, while those in the no-treatment control condition did not.

Figure 6.2 Hypothetical Results From a Study Including Treatment, No-Treatment, and Placebo Conditions

Fortunately, there are several solutions to this problem. One is to include a placebo control condition , in which participants receive a placebo that looks much like the treatment but lacks the active ingredient or element thought to be responsible for the treatment’s effectiveness. When participants in a treatment condition take a pill, for example, then those in a placebo control condition would take an identical-looking pill that lacks the active ingredient in the treatment (a “sugar pill”). In research on psychotherapy effectiveness, the placebo might involve going to a psychotherapist and talking in an unstructured way about one’s problems. The idea is that if participants in both the treatment and the placebo control groups expect to improve, then any improvement in the treatment group over and above that in the placebo control group must have been caused by the treatment and not by participants’ expectations. This is what is shown by a comparison of the two outer bars in Figure 6.2 “Hypothetical Results From a Study Including Treatment, No-Treatment, and Placebo Conditions” .

Of course, the principle of informed consent requires that participants be told that they will be assigned to either a treatment or a placebo control condition—even though they cannot be told which until the experiment ends. In many cases the participants who had been in the control condition are then offered an opportunity to have the real treatment. An alternative approach is to use a waitlist control condition , in which participants are told that they will receive the treatment but must wait until the participants in the treatment condition have already received it. This allows researchers to compare participants who have received the treatment with participants who are not currently receiving it but who still expect to improve (eventually). A final solution to the problem of placebo effects is to leave out the control condition completely and compare any new treatment with the best available alternative treatment. For example, a new treatment for simple phobia could be compared with standard exposure therapy. Because participants in both conditions receive a treatment, their expectations about improvement should be similar. This approach also makes sense because once there is an effective treatment, the interesting question about a new treatment is not simply “Does it work?” but “Does it work better than what is already available?”

The Powerful Placebo

Many people are not surprised that placebos can have a positive effect on disorders that seem fundamentally psychological, including depression, anxiety, and insomnia. However, placebos can also have a positive effect on disorders that most people think of as fundamentally physiological. These include asthma, ulcers, and warts (Shapiro & Shapiro, 1999). There is even evidence that placebo surgery—also called “sham surgery”—can be as effective as actual surgery.

Medical researcher J. Bruce Moseley and his colleagues conducted a study on the effectiveness of two arthroscopic surgery procedures for osteoarthritis of the knee (Moseley et al., 2002). The control participants in this study were prepped for surgery, received a tranquilizer, and even received three small incisions in their knees. But they did not receive the actual arthroscopic surgical procedure. The surprising result was that all participants improved in terms of both knee pain and function, and the sham surgery group improved just as much as the treatment groups. According to the researchers, “This study provides strong evidence that arthroscopic lavage with or without débridement [the surgical procedures used] is not better than and appears to be equivalent to a placebo procedure in improving knee pain and self-reported function” (p. 85).

Research has shown that patients with osteoarthritis of the knee who receive a “sham surgery” experience reductions in pain and improvement in knee function similar to those of patients who receive a real surgery.

Army Medicine – Surgery – CC BY 2.0.

Within-Subjects Experiments

In a within-subjects experiment , each participant is tested under all conditions. Consider an experiment on the effect of a defendant’s physical attractiveness on judgments of his guilt. Again, in a between-subjects experiment, one group of participants would be shown an attractive defendant and asked to judge his guilt, and another group of participants would be shown an unattractive defendant and asked to judge his guilt. In a within-subjects experiment, however, the same group of participants would judge the guilt of both an attractive and an unattractive defendant.

The primary advantage of this approach is that it provides maximum control of extraneous participant variables. Participants in all conditions have the same mean IQ, same socioeconomic status, same number of siblings, and so on—because they are the very same people. Within-subjects experiments also make it possible to use statistical procedures that remove the effect of these extraneous participant variables on the dependent variable and therefore make the data less “noisy” and the effect of the independent variable easier to detect. We will look more closely at this idea later in the book.

Carryover Effects and Counterbalancing

The primary disadvantage of within-subjects designs is that they can result in carryover effects. A carryover effect is an effect of being tested in one condition on participants’ behavior in later conditions. One type of carryover effect is a practice effect , where participants perform a task better in later conditions because they have had a chance to practice it. Another type is a fatigue effect , where participants perform a task worse in later conditions because they become tired or bored. Being tested in one condition can also change how participants perceive stimuli or interpret their task in later conditions. This is called a context effect . For example, an average-looking defendant might be judged more harshly when participants have just judged an attractive defendant than when they have just judged an unattractive defendant. Within-subjects experiments also make it easier for participants to guess the hypothesis. For example, a participant who is asked to judge the guilt of an attractive defendant and then is asked to judge the guilt of an unattractive defendant is likely to guess that the hypothesis is that defendant attractiveness affects judgments of guilt. This could lead the participant to judge the unattractive defendant more harshly because he thinks this is what he is expected to do. Or it could make participants judge the two defendants similarly in an effort to be “fair.”

Carryover effects can be interesting in their own right. (Does the attractiveness of one person depend on the attractiveness of other people that we have seen recently?) But when they are not the focus of the research, carryover effects can be problematic. Imagine, for example, that participants judge the guilt of an attractive defendant and then judge the guilt of an unattractive defendant. If they judge the unattractive defendant more harshly, this might be because of his unattractiveness. But it could be instead that they judge him more harshly because they are becoming bored or tired. In other words, the order of the conditions is a confounding variable. The attractive condition is always the first condition and the unattractive condition the second. Thus any difference between the conditions in terms of the dependent variable could be caused by the order of the conditions and not the independent variable itself.

There is a solution to the problem of order effects, however, that can be used in many situations. It is counterbalancing , which means testing different participants in different orders. For example, some participants would be tested in the attractive defendant condition followed by the unattractive defendant condition, and others would be tested in the unattractive condition followed by the attractive condition. With three conditions, there would be six different orders (ABC, ACB, BAC, BCA, CAB, and CBA), so some participants would be tested in each of the six orders. With counterbalancing, participants are assigned to orders randomly, using the techniques we have already discussed. Thus random assignment plays an important role in within-subjects designs just as in between-subjects designs. Here, instead of randomly assigning to conditions, they are randomly assigned to different orders of conditions. In fact, it can safely be said that if a study does not involve random assignment in one form or another, it is not an experiment.

There are two ways to think about what counterbalancing accomplishes. One is that it controls the order of conditions so that it is no longer a confounding variable. Instead of the attractive condition always being first and the unattractive condition always being second, the attractive condition comes first for some participants and second for others. Likewise, the unattractive condition comes first for some participants and second for others. Thus any overall difference in the dependent variable between the two conditions cannot have been caused by the order of conditions. A second way to think about what counterbalancing accomplishes is that if there are carryover effects, it makes it possible to detect them. One can analyze the data separately for each order to see whether it had an effect.

When 9 Is “Larger” Than 221

Researcher Michael Birnbaum has argued that the lack of context provided by between-subjects designs is often a bigger problem than the context effects created by within-subjects designs. To demonstrate this, he asked one group of participants to rate how large the number 9 was on a 1-to-10 rating scale and another group to rate how large the number 221 was on the same 1-to-10 rating scale (Birnbaum, 1999). Participants in this between-subjects design gave the number 9 a mean rating of 5.13 and the number 221 a mean rating of 3.10. In other words, they rated 9 as larger than 221! According to Birnbaum, this is because participants spontaneously compared 9 with other one-digit numbers (in which case it is relatively large) and compared 221 with other three-digit numbers (in which case it is relatively small).

Simultaneous Within-Subjects Designs

So far, we have discussed an approach to within-subjects designs in which participants are tested in one condition at a time. There is another approach, however, that is often used when participants make multiple responses in each condition. Imagine, for example, that participants judge the guilt of 10 attractive defendants and 10 unattractive defendants. Instead of having people make judgments about all 10 defendants of one type followed by all 10 defendants of the other type, the researcher could present all 20 defendants in a sequence that mixed the two types. The researcher could then compute each participant’s mean rating for each type of defendant. Or imagine an experiment designed to see whether people with social anxiety disorder remember negative adjectives (e.g., “stupid,” “incompetent”) better than positive ones (e.g., “happy,” “productive”). The researcher could have participants study a single list that includes both kinds of words and then have them try to recall as many words as possible. The researcher could then count the number of each type of word that was recalled. There are many ways to determine the order in which the stimuli are presented, but one common way is to generate a different random order for each participant.

Between-Subjects or Within-Subjects?

Almost every experiment can be conducted using either a between-subjects design or a within-subjects design. This means that researchers must choose between the two approaches based on their relative merits for the particular situation.

Between-subjects experiments have the advantage of being conceptually simpler and requiring less testing time per participant. They also avoid carryover effects without the need for counterbalancing. Within-subjects experiments have the advantage of controlling extraneous participant variables, which generally reduces noise in the data and makes it easier to detect a relationship between the independent and dependent variables.

A good rule of thumb, then, is that if it is possible to conduct a within-subjects experiment (with proper counterbalancing) in the time that is available per participant—and you have no serious concerns about carryover effects—this is probably the best option. If a within-subjects design would be difficult or impossible to carry out, then you should consider a between-subjects design instead. For example, if you were testing participants in a doctor’s waiting room or shoppers in line at a grocery store, you might not have enough time to test each participant in all conditions and therefore would opt for a between-subjects design. Or imagine you were trying to reduce people’s level of prejudice by having them interact with someone of another race. A within-subjects design with counterbalancing would require testing some participants in the treatment condition first and then in a control condition. But if the treatment works and reduces people’s level of prejudice, then they would no longer be suitable for testing in the control condition. This is true for many designs that involve a treatment meant to produce long-term change in participants’ behavior (e.g., studies testing the effectiveness of psychotherapy). Clearly, a between-subjects design would be necessary here.

Remember also that using one type of design does not preclude using the other type in a different study. There is no reason that a researcher could not use both a between-subjects design and a within-subjects design to answer the same research question. In fact, professional researchers often do exactly this.

Key Takeaways

• Experiments can be conducted using either between-subjects or within-subjects designs. Deciding which to use in a particular situation requires careful consideration of the pros and cons of each approach.
• Random assignment to conditions in between-subjects experiments or to orders of conditions in within-subjects experiments is a fundamental element of experimental research. Its purpose is to control extraneous variables so that they do not become confounding variables.
• Experimental research on the effectiveness of a treatment requires both a treatment condition and a control condition, which can be a no-treatment control condition, a placebo control condition, or a waitlist control condition. Experimental treatments can also be compared with the best available alternative.

Discussion: For each of the following topics, list the pros and cons of a between-subjects and within-subjects design and decide which would be better.

• You want to test the relative effectiveness of two training programs for running a marathon.
• Using photographs of people as stimuli, you want to see if smiling people are perceived as more intelligent than people who are not smiling.
• In a field experiment, you want to see if the way a panhandler is dressed (neatly vs. sloppily) affects whether or not passersby give him any money.
• You want to see if concrete nouns (e.g., dog ) are recalled better than abstract nouns (e.g., truth ).
• Discussion: Imagine that an experiment shows that participants who receive psychodynamic therapy for a dog phobia improve more than participants in a no-treatment control group. Explain a fundamental problem with this research design and at least two ways that it might be corrected.

Birnbaum, M. H. (1999). How to show that 9 > 221: Collect judgments in a between-subjects design. Psychological Methods, 4 , 243–249.

Moseley, J. B., O’Malley, K., Petersen, N. J., Menke, T. J., Brody, B. A., Kuykendall, D. H., … Wray, N. P. (2002). A controlled trial of arthroscopic surgery for osteoarthritis of the knee. The New England Journal of Medicine, 347 , 81–88.

Price, D. D., Finniss, D. G., & Benedetti, F. (2008). A comprehensive review of the placebo effect: Recent advances and current thought. Annual Review of Psychology, 59 , 565–590.

Shapiro, A. K., & Shapiro, E. (1999). The powerful placebo: From ancient priest to modern physician . Baltimore, MD: Johns Hopkins University Press.

Research Methods in Psychology Copyright © 2016 by University of Minnesota is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License , except where otherwise noted.

If you're seeing this message, it means we're having trouble loading external resources on our website.

If you're behind a web filter, please make sure that the domains *.kastatic.org and *.kasandbox.org are unblocked.

To log in and use all the features of Khan Academy, please enable JavaScript in your browser.

Statistics and probability

Course: statistics and probability   >   unit 6.

• Introduction to experiment design
• Matched pairs experiment design
• The language of experiments
• Principles of experiment design
• Experiment designs

Random sampling vs. random assignment (scope of inference)

• (Choice A)   Just the residents involved in Hilary's study. A Just the residents involved in Hilary's study.
• (Choice B)   All residents in Hilary's town. B All residents in Hilary's town.
• (Choice C)   All residents in Hilary's country. C All residents in Hilary's country.
• (Choice A)   Yes A Yes
• (Choice B)   No B No
• (Choice A)   Just the residents in Hilary's study. A Just the residents in Hilary's study.

Want to join the conversation?

If you're seeing this message, it means we're having trouble loading external resources on our website.

If you're behind a web filter, please make sure that the domains *.kastatic.org and *.kasandbox.org are unblocked.

To log in and use all the features of Khan Academy, please enable JavaScript in your browser.

Statistics and probability

Course: statistics and probability   >   unit 6.

• Picking fairly
• Using probability to make fair decisions
• Techniques for generating a simple random sample
• Simple random samples
• Techniques for random sampling and avoiding bias
• Sampling methods

Sampling methods review

• Samples and surveys

What are sampling methods?

Bad ways to sample.

• (Choice A)   Convenience sampling A Convenience sampling
• (Choice B)   Voluntary response sampling B Voluntary response sampling

Good ways to sample

• (Choice A)   Simple random sampling A Simple random sampling
• (Choice B)   Stratified random sampling B Stratified random sampling
• (Choice C)   Cluster random sampling C Cluster random sampling
• (Choice D)   Systematic random sampling D Systematic random sampling

Want to join the conversation?

• Upvote Button navigates to signup page
• Downvote Button navigates to signup page
• Flag Button navigates to signup page

• Bipolar Disorder
• Therapy Center
• When To See a Therapist
• Types of Therapy
• Best Online Therapy
• Best Couples Therapy
• Best Family Therapy
• Managing Stress
• Sleep and Dreaming
• Understanding Emotions
• Self-Improvement
• Healthy Relationships
• Student Resources
• Personality Types
• Guided Meditations
• Verywell Mind Insights
• 2023 Verywell Mind 25
• Mental Health in the Classroom
• Editorial Process
• Meet Our Review Board
• Crisis Support

The Definition of Random Assignment According to Psychology

Kendra Cherry, MS, is a psychosocial rehabilitation specialist, psychology educator, and author of the "Everything Psychology Book."

Emily is a board-certified science editor who has worked with top digital publishing brands like Voices for Biodiversity, Study.com, GoodTherapy, Vox, and Verywell.

Materio / Getty Images

Random assignment refers to the use of chance procedures in psychology experiments to ensure that each participant has the same opportunity to be assigned to any given group in a study to eliminate any potential bias in the experiment at the outset. Participants are randomly assigned to different groups, such as the treatment group versus the control group. In clinical research, randomized clinical trials are known as the gold standard for meaningful results.

Simple random assignment techniques might involve tactics such as flipping a coin, drawing names out of a hat, rolling dice, or assigning random numbers to a list of participants. It is important to note that random assignment differs from random selection .

While random selection refers to how participants are randomly chosen from a target population as representatives of that population, random assignment refers to how those chosen participants are then assigned to experimental groups.

Random Assignment In Research

To determine if changes in one variable will cause changes in another variable, psychologists must perform an experiment. Random assignment is a critical part of the experimental design that helps ensure the reliability of the study outcomes.

Researchers often begin by forming a testable hypothesis predicting that one variable of interest will have some predictable impact on another variable.

The variable that the experimenters will manipulate in the experiment is known as the independent variable , while the variable that they will then measure for different outcomes is known as the dependent variable. While there are different ways to look at relationships between variables, an experiment is the best way to get a clear idea if there is a cause-and-effect relationship between two or more variables.

Once researchers have formulated a hypothesis, conducted background research, and chosen an experimental design, it is time to find participants for their experiment. How exactly do researchers decide who will be part of an experiment? As mentioned previously, this is often accomplished through something known as random selection.

Random Selection

In order to generalize the results of an experiment to a larger group, it is important to choose a sample that is representative of the qualities found in that population. For example, if the total population is 60% female and 40% male, then the sample should reflect those same percentages.

Choosing a representative sample is often accomplished by randomly picking people from the population to be participants in a study. Random selection means that everyone in the group stands an equal chance of being chosen to minimize any bias. Once a pool of participants has been selected, it is time to assign them to groups.

By randomly assigning the participants into groups, the experimenters can be fairly sure that each group will have the same characteristics before the independent variable is applied.

Participants might be randomly assigned to the control group , which does not receive the treatment in question. The control group may receive a placebo or receive the standard treatment. Participants may also be randomly assigned to the experimental group , which receives the treatment of interest. In larger studies, there can be multiple treatment groups for comparison.

There are simple methods of random assignment, like rolling the die. However, there are more complex techniques that involve random number generators to remove any human error.

There can also be random assignment to groups with pre-established rules or parameters. For example, if you want to have an equal number of men and women in each of your study groups, you might separate your sample into two groups (by sex) before randomly assigning each of those groups into the treatment group and control group.

Random assignment is essential because it increases the likelihood that the groups are the same at the outset. With all characteristics being equal between groups, other than the application of the independent variable, any differences found between group outcomes can be more confidently attributed to the effect of the intervention.

Example of Random Assignment

Imagine that a researcher is interested in learning whether or not drinking caffeinated beverages prior to an exam will improve test performance. After randomly selecting a pool of participants, each person is randomly assigned to either the control group or the experimental group.

The participants in the control group consume a placebo drink prior to the exam that does not contain any caffeine. Those in the experimental group, on the other hand, consume a caffeinated beverage before taking the test.

Participants in both groups then take the test, and the researcher compares the results to determine if the caffeinated beverage had any impact on test performance.

A Word From Verywell

Random assignment plays an important role in the psychology research process. Not only does this process help eliminate possible sources of bias, but it also makes it easier to generalize the results of a tested sample of participants to a larger population.

Random assignment helps ensure that members of each group in the experiment are the same, which means that the groups are also likely more representative of what is present in the larger population of interest. Through the use of this technique, psychology researchers are able to study complex phenomena and contribute to our understanding of the human mind and behavior.

Lin Y, Zhu M, Su Z. The pursuit of balance: An overview of covariate-adaptive randomization techniques in clinical trials . Contemp Clin Trials. 2015;45(Pt A):21-25. doi:10.1016/j.cct.2015.07.011

Sullivan L. Random assignment versus random selection . In: The SAGE Glossary of the Social and Behavioral Sciences. SAGE Publications, Inc.; 2009. doi:10.4135/9781412972024.n2108

Alferes VR. Methods of Randomization in Experimental Design . SAGE Publications, Inc.; 2012. doi:10.4135/9781452270012

Nestor PG, Schutt RK. Research Methods in Psychology: Investigating Human Behavior. (2nd Ed.). SAGE Publications, Inc.; 2015.

By Kendra Cherry, MSEd Kendra Cherry, MS, is a psychosocial rehabilitation specialist, psychology educator, and author of the "Everything Psychology Book."

LEARN STATISTICS EASILY

Learn Data Analysis Now!

Understanding Random Sampling: Essential Techniques in Data Analysis

Random sampling in statistics is a technique for selecting a subset of individuals from a larger population where each individual has an equal chance of being chosen. This method ensures representative samples, minimizes bias and allows for reliable inferences about the population based on the sample data.

Definition and Importance of Random Sampling

Random sampling is fundamental in data analysis, statistics, and broader scientific research. It refers to the technique of selecting individuals or elements from a population such that each individual has an equal probability of being chosen. This method is essential as it ensures a representative sample, thereby eliminating bias and enabling researchers to draw valid conclusions about the whole population based on the sample data.

The importance of random sampling in data analysis cannot be overstated. Instead, it forms the basis of hypothesis testing, inferential statistics, and prediction modeling. Without random sampling, we risk introducing selection bias into our study, which can lead to inaccurate conclusions and misleading results. The strength of random sampling lies in its ability to mirror the characteristics of the whole population within the sample, enhancing the reliability and validity of the analysis.

• In random sampling, every member of a population has an equal chance to be chosen as part of the sample.
• It forms the basis of hypothesis testing, inferential statistics, and prediction modeling.
• Simple random sampling, the most basic form, is adequate when the population is homogeneous.
• Stratified random sampling divides the population into subgroups, ensuring sufficient representation.
• Systematic random sampling selects individuals at regular intervals from the population.

Types of Random Sampling

Simple Random Sampling  is the most basic type of random sampling. Each population element has an equal chance of being selected in this method. The selection is often made through a random process, such as using a random number generator or drawing names from a hat. This method is most effective when the population is homogeneous, i.e., when the characteristics of individuals don’t significantly vary. Imagine a small town that wants to survey residents’ satisfaction with local services. They could use simple random sampling by assigning each resident a number and then using a random number generator to select 100 residents to participate in the survey.

Stratified Random Sampling  is a technique used when the population is not homogeneous. The population is categorized into strata (or subgroups) based on specific characteristics such as age, gender, or geographic location. Then, random sampling is applied within each stratum to select the individuals. This method ensures that each subgroup is adequately represented in the sample. Suppose a national clothing retailer wants to understand customer satisfaction across different age groups. They could divide their customer base into distinct age groups, such as 18-29, 30-39, 40-49, etc., and then perform simple random sampling within these strata to ensure that all age groups are adequately represented.

Cluster Random Sampling  involves dividing the population into separate groups or clusters, usually based on geographic location. A random sample of clusters is selected, and all individuals within these chosen clusters are included. This method is often used when conducting simple or stratified sampling is costly or impractical. Consider a situation where a government health agency wants to study lifestyle habits nationwide. It would be impractical and expensive to randomly sample individuals from the entire country. Instead, they could use cluster sampling. They might divide the country into clusters by postcode and then randomly select a few postcodes. Every resident within the selected postcodes would be included in the study.

Systematic Random Sampling  involves selecting individuals at regular intervals from the population. The first individual is chosen randomly, and then every nth is selected. This method is often used when a complete list of the population is available, and it’s important to note that it requires the assumption that the list is not patterned in any way. Suppose a university wants to assess the effectiveness of its new online learning platform. They could use systematic random sampling by alphabetizing all students and selecting every 10th student for a survey. This method would provide a sample spread evenly across the entire student population.

Challenges and Misconceptions about Random Sampling

Despite the importance of random sampling, several challenges and misconceptions can hamper its effective implementation.

One common misconception is that random sampling produces a sample that perfectly represents the population. While random sampling is designed to minimize bias and increase the likelihood of representativeness, it does not guarantee it. There’s always a chance that the sample might not accurately reflect the population due to random variation.

Another challenge is the practical implementation of random sampling. Often, having a complete population list or randomly selecting individuals may be impossible. For instance, respondents self-select to participate in online surveys, which may introduce bias.

Furthermore, there is a typical misconception that a larger sample is always better. While it’s true that increasing the sample size can often decrease the margin of error and increase the confidence level, it also increases the time and cost of data collection and analysis. Therefore, balancing the need for precision with practical considerations is crucial.

In summary, while random sampling is a cornerstone of statistical and data analysis, it has challenges and misconceptions. Understanding these can help researchers and analysts better design and implement their studies for robust, reliable, and meaningful results.

Recommended Articles

Want to explore more about data analysis and statistics? Don’t stop at random sampling. Our blog features many articles covering various topics that will deepen your understanding and enhance your skills. Whether starting or looking to advance your knowledge, we’ve got you covered. Look at our other posts today and continue your learning journey with us!

• Unraveling Sampling Bias: A Comprehensive Guide
• Selection Bias in Data Analysis: Understanding the Intricacies

Frequently Asked Questions (FAQs)

Q1: What are the 4 types of random sampling?  The four main types of random sampling are Simple, Stratified, Cluster, and Systematic Random Sampling. Each has its unique application depending on the nature of the population and the research question.

Q2: Why is random sampling used?  Random sampling is used to pick a representative sample from a larger population, ensuring each individual has an equal chance of being chosen. This minimizes selection bias, making inferences about the population more accurate.

Q3: What is a random sample in statistics?  A random sample in statistics is a subset of individuals or data points selected from a larger population. Each individual or point has an equal probability of being chosen.

Q4: How is random sampling done?  Random sampling is done by assigning each individual in the population a unique identifier and then using a random process (like a random number generator) to select a subset of individuals.

Q5: Which random sampling method is best?  The “best” random sampling method depends on the specifics of the study, including the nature of the population, the research question, and practical considerations. Each method has its strengths and weaknesses.

Q6: How do you decide on what type of sampling to use?  The choice of sampling method depends on several factors, including the research question, the nature of the population, the availability of a complete list of the population, and practical constraints such as time and cost.

Q7: What are the challenges of random sampling?  Challenges of random sampling include practical implementation issues, the potential for nonresponse bias, and the misconception that a larger sample is always better or more representative.

Q8: Can random sampling eliminate all bias?  While random sampling can help reduce selection bias, it does not stop all types of bias. For example, it can’t correct measurement errors or biases in data collection.

Q9: How does stratified random sampling differ from simple random sampling?  Stratified random sampling is distinct from simple random sampling. It first divides the population into different subgroups, or strata, based on specific characteristics. Then, simple random sampling is performed within each subset. This ensures that each subgroup is adequately represented in the sample, which can be especially useful when the population is heterogeneous.

Q10: What is an example of cluster random sampling?  Cluster random sampling involves dividing the population into clusters and then randomly selecting a few clusters for study. For instance, a researcher studying educational practices might divide a country into clusters by school districts, then randomly select a few districts. All schools within these selected districts would be included in the study.

Similar Posts

Applied Statistics: Data Analysis

If you’re struggling with statistics while analyzing data for your projects, this is your ultimate solution for Data Analysis!

APA Style T-Test Reporting Guide

How to report t-test in APA style? Learn now! Tips, examples, and effect size calculation included. Learn how to report t-test results.

Can Standard Deviations Be Negative? Data Analysis Understanding

Discover why standard deviations can’t be negative and deepen your understanding of data analysis in this insightful article.

Non-Parametric Statistics: A Comprehensive Guide

Unlock the potential of Non-Parametric Statistics to analyze complex data with our guide, offering insights into flexible data analysis.

What Does The P-Value Mean?

Discover what the p-value means is in statistics, its role in determining significance, and how it affects your data analysis!

Descriptive vs. Inferential Statistics: A Comprehensive Guide

Discover the critical differences between descriptive vs. inferential statistics, their applications, and how to effectively use them.

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

Psychological Research

Drawing conclusions from statistics, learning objectives.

• Describe the role of random sampling and random assignment in drawing cause-and-effect conclusions

Generalizability

Figure 1 . Generalizability is an important research consideration: The results of studies with widely representative samples are more likely to generalize to the population. [Image: Barnacles Budget Accommodation]

One limitation to the study mentioned previously about the babies choosing the “helper” toy is that the conclusion only applies to the 16 infants in the study. We don’t know much about how those 16 infants were selected. Suppose we want to select a subset of individuals (a sample ) from a much larger group of individuals (the population ) in such a way that conclusions from the sample can be generalized to the larger population. This is the question faced by pollsters every day.

Example 1 : The General Social Survey (GSS) is a survey on societal trends conducted every other year in the United States. Based on a sample of about 2,000 adult Americans, researchers make claims about what percentage of the U.S. population consider themselves to be “liberal,” what percentage consider themselves “happy,” what percentage feel “rushed” in their daily lives, and many other issues. The key to making these claims about the larger population of all American adults lies in how the sample is selected. The goal is to select a sample that is representative of the population, and a common way to achieve this goal is to select a random sample that gives every member of the population an equal chance of being selected for the sample. In its simplest form, random sampling involves numbering every member of the population and then using a computer to randomly select the subset to be surveyed. Most polls don’t operate exactly like this, but they do use probability-based sampling methods to select individuals from nationally representative panels.

In 2004, the GSS reported that 817 of 977 respondents (or 83.6%) indicated that they always or sometimes feel rushed. This is a clear majority, but we again need to consider variation due to random sampling . Fortunately, we can use the same probability model we did in the previous example to investigate the probable size of this error. (Note, we can use the coin-tossing model when the actual population size is much, much larger than the sample size, as then we can still consider the probability to be the same for every individual in the sample.) This probability model predicts that the sample result will be within 3 percentage points of the population value (roughly 1 over the square root of the sample size, the margin of error ). A statistician would conclude, with 95% confidence, that between 80.6% and 86.6% of all adult Americans in 2004 would have responded that they sometimes or always feel rushed.

The key to the margin of error is that when we use a probability sampling method, we can make claims about how often (in the long run, with repeated random sampling) the sample result would fall within a certain distance from the unknown population value by chance (meaning by random sampling variation) alone. Conversely, non-random samples are often suspect to bias, meaning the sampling method systematically over-represents some segments of the population and under-represents others. We also still need to consider other sources of bias, such as individuals not responding honestly. These sources of error are not measured by the margin of error.

Cause and Effect

In many research studies, the primary question of interest concerns differences between groups. Then the question becomes how were the groups formed (e.g., selecting people who already drink coffee vs. those who don’t). In some studies, the researchers actively form the groups themselves. But then we have a similar question—could any differences we observe in the groups be an artifact of that group-formation process? Or maybe the difference we observe in the groups is so large that we can discount a “fluke” in the group-formation process as a reasonable explanation for what we find?

Example 2 : A psychology study investigated whether people tend to display more creativity when they are thinking about intrinsic (internal) or extrinsic (external) motivations (Ramsey & Schafer, 2002, based on a study by Amabile, 1985). The subjects were 47 people with extensive experience with creative writing. Subjects began by answering survey questions about either intrinsic motivations for writing (such as the pleasure of self-expression) or extrinsic motivations (such as public recognition). Then all subjects were instructed to write a haiku, and those poems were evaluated for creativity by a panel of judges. The researchers conjectured beforehand that subjects who were thinking about intrinsic motivations would display more creativity than subjects who were thinking about extrinsic motivations. The creativity scores from the 47 subjects in this study are displayed in Figure 2, where higher scores indicate more creativity.

Figure 2 . Creativity scores separated by type of motivation.

In this example, the key question is whether the type of motivation affects creativity scores. In particular, do subjects who were asked about intrinsic motivations tend to have higher creativity scores than subjects who were asked about extrinsic motivations?

Figure 2 reveals that both motivation groups saw considerable variability in creativity scores, and these scores have considerable overlap between the groups. In other words, it’s certainly not always the case that those with extrinsic motivations have higher creativity than those with intrinsic motivations, but there may still be a statistical tendency in this direction. (Psychologist Keith Stanovich (2013) refers to people’s difficulties with thinking about such probabilistic tendencies as “the Achilles heel of human cognition.”)

The mean creativity score is 19.88 for the intrinsic group, compared to 15.74 for the extrinsic group, which supports the researchers’ conjecture. Yet comparing only the means of the two groups fails to consider the variability of creativity scores in the groups. We can measure variability with statistics using, for instance, the standard deviation: 5.25 for the extrinsic group and 4.40 for the intrinsic group. The standard deviations tell us that most of the creativity scores are within about 5 points of the mean score in each group. We see that the mean score for the intrinsic group lies within one standard deviation of the mean score for extrinsic group. So, although there is a tendency for the creativity scores to be higher in the intrinsic group, on average, the difference is not extremely large.

We again want to consider possible explanations for this difference. The study only involved individuals with extensive creative writing experience. Although this limits the population to which we can generalize, it does not explain why the mean creativity score was a bit larger for the intrinsic group than for the extrinsic group. Maybe women tend to receive higher creativity scores? Here is where we need to focus on how the individuals were assigned to the motivation groups. If only women were in the intrinsic motivation group and only men in the extrinsic group, then this would present a problem because we wouldn’t know if the intrinsic group did better because of the different type of motivation or because they were women. However, the researchers guarded against such a problem by randomly assigning the individuals to the motivation groups. Like flipping a coin, each individual was just as likely to be assigned to either type of motivation. Why is this helpful? Because this random assignment tends to balance out all the variables related to creativity we can think of, and even those we don’t think of in advance, between the two groups. So we should have a similar male/female split between the two groups; we should have a similar age distribution between the two groups; we should have a similar distribution of educational background between the two groups; and so on. Random assignment should produce groups that are as similar as possible except for the type of motivation, which presumably eliminates all those other variables as possible explanations for the observed tendency for higher scores in the intrinsic group.

But does this always work? No, so by “luck of the draw” the groups may be a little different prior to answering the motivation survey. So then the question is, is it possible that an unlucky random assignment is responsible for the observed difference in creativity scores between the groups? In other words, suppose each individual’s poem was going to get the same creativity score no matter which group they were assigned to, that the type of motivation in no way impacted their score. Then how often would the random-assignment process alone lead to a difference in mean creativity scores as large (or larger) than 19.88 – 15.74 = 4.14 points?

We again want to apply to a probability model to approximate a p-value , but this time the model will be a bit different. Think of writing everyone’s creativity scores on an index card, shuffling up the index cards, and then dealing out 23 to the extrinsic motivation group and 24 to the intrinsic motivation group, and finding the difference in the group means. We (better yet, the computer) can repeat this process over and over to see how often, when the scores don’t change, random assignment leads to a difference in means at least as large as 4.41. Figure 3 shows the results from 1,000 such hypothetical random assignments for these scores.

Figure 3 . Differences in group means under random assignment alone.

Only 2 of the 1,000 simulated random assignments produced a difference in group means of 4.41 or larger. In other words, the approximate p-value is 2/1000 = 0.002. This small p-value indicates that it would be very surprising for the random assignment process alone to produce such a large difference in group means. Therefore, as with Example 2, we have strong evidence that focusing on intrinsic motivations tends to increase creativity scores, as compared to thinking about extrinsic motivations.

Notice that the previous statement implies a cause-and-effect relationship between motivation and creativity score; is such a strong conclusion justified? Yes, because of the random assignment used in the study. That should have balanced out any other variables between the two groups, so now that the small p-value convinces us that the higher mean in the intrinsic group wasn’t just a coincidence, the only reasonable explanation left is the difference in the type of motivation. Can we generalize this conclusion to everyone? Not necessarily—we could cautiously generalize this conclusion to individuals with extensive experience in creative writing similar the individuals in this study, but we would still want to know more about how these individuals were selected to participate.

Figure 4 . Researchers employ the scientific method that involves a great deal of statistical thinking: generate a hypothesis –> design a study to test that hypothesis –> conduct the study –> analyze the data –> report the results. [Image: widdowquinn]

Statistical thinking involves the careful design of a study to collect meaningful data to answer a focused research question, detailed analysis of patterns in the data, and drawing conclusions that go beyond the observed data. Random sampling is paramount to generalizing results from our sample to a larger population, and random assignment is key to drawing cause-and-effect conclusions. With both kinds of randomness, probability models help us assess how much random variation we can expect in our results, in order to determine whether our results could happen by chance alone and to estimate a margin of error.

So where does this leave us with regard to the coffee study mentioned previously (the Freedman, Park, Abnet, Hollenbeck, & Sinha, 2012 found that men who drank at least six cups of coffee a day had a 10% lower chance of dying (women 15% lower) than those who drank none)? We can answer many of the questions:

• This was a 14-year study conducted by researchers at the National Cancer Institute.
• The results were published in the June issue of the New England Journal of Medicine , a respected, peer-reviewed journal.
• The study reviewed coffee habits of more than 402,000 people ages 50 to 71 from six states and two metropolitan areas. Those with cancer, heart disease, and stroke were excluded at the start of the study. Coffee consumption was assessed once at the start of the study.
• About 52,000 people died during the course of the study.
• People who drank between two and five cups of coffee daily showed a lower risk as well, but the amount of reduction increased for those drinking six or more cups.
• The sample sizes were fairly large and so the p-values are quite small, even though percent reduction in risk was not extremely large (dropping from a 12% chance to about 10%–11%).
• Whether coffee was caffeinated or decaffeinated did not appear to affect the results.
• This was an observational study, so no cause-and-effect conclusions can be drawn between coffee drinking and increased longevity, contrary to the impression conveyed by many news headlines about this study. In particular, it’s possible that those with chronic diseases don’t tend to drink coffee.

This study needs to be reviewed in the larger context of similar studies and consistency of results across studies, with the constant caution that this was not a randomized experiment. Whereas a statistical analysis can still “adjust” for other potential confounding variables, we are not yet convinced that researchers have identified them all or completely isolated why this decrease in death risk is evident. Researchers can now take the findings of this study and develop more focused studies that address new questions.

Explore these outside resources to learn more about applied statistics:

• Video about p-values:  P-Value Extravaganza
• Interactive web applets for teaching and learning statistics
• Inter-university Consortium for Political and Social Research  where you can find and analyze data.
• The Consortium for the Advancement of Undergraduate Statistics

Think It Over

• Find a recent research article in your field and answer the following: What was the primary research question? How were individuals selected to participate in the study? Were summary results provided? How strong is the evidence presented in favor or against the research question? Was random assignment used? Summarize the main conclusions from the study, addressing the issues of statistical significance, statistical confidence, generalizability, and cause and effect. Do you agree with the conclusions drawn from this study, based on the study design and the results presented?
• Is it reasonable to use a random sample of 1,000 individuals to draw conclusions about all U.S. adults? Explain why or why not.

cause-and-effect: related to whether we say one variable is causing changes in the other variable, versus other variables that may be related to these two variables. generalizability : related to whether the results from the sample can be generalized to a larger population. margin of error : the expected amount of random variation in a statistic; often defined for 95% confidence level. population : a larger collection of individuals that we would like to generalize our results to. p-value : the probability of observing a particular outcome in a sample, or more extreme, under a conjecture about the larger population or process. random assignment : using a probability-based method to divide a sample into treatment groups. random sampling : using a probability-based method to select a subset of individuals for the sample from the population. sample : the collection of individuals on which we collect data.

Contribute!

Improve this page Learn More

• Modification, adaptation, and original content. Authored by : Pat Carroll and Lumen Learning. Provided by : Lumen Learning. License : CC BY: Attribution
• Statistical Thinking. Authored by : Beth Chance and Allan Rossman, California Polytechnic State University, San Luis Obispo. Provided by : Noba. Located at : http://nobaproject.com/modules/statistical-thinking . License : CC BY-NC-SA: Attribution-NonCommercial-ShareAlike
• The Replication Crisis. Authored by : Colin Thomas William. Provided by : Ivy Tech Community College. License : CC BY: Attribution

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, automatically generate references for free.

• Knowledge Base
• Methodology
• Simple Random Sampling | Definition, Steps & Examples

Simple Random Sampling | Definition, Steps & Examples

Published on 3 May 2022 by Lauren Thomas . Revised on 18 December 2023.

A simple random sample is a randomly selected subset of a population . In this sampling method, each member of the population has an exactly equal chance of being selected, minimising the risk of selection bias .

This method is the most straightforward of all the probability sampling methods , since it only involves a single random selection and requires little advance knowledge about the population. Because it uses randomisation, any research performed on this sample should have high internal and external validity.

Table of contents

When to use simple random sampling, how to perform simple random sampling, frequently asked questions about simple random sampling.

Simple random sampling is used to make statistical inferences about a population. It helps ensure high internal validity : randomisation is the best method to reduce the impact of potential confounding variables .

In addition, with a large enough sample size, a simple random sample has high external validity : it represents the characteristics of the larger population.

However, simple random sampling can be challenging to implement in practice. To use this method, there are some prerequisites:

• You have a complete list of every member of the population.
• You can contact or access each member of the population if they are selected.
• You have the time and resources to collect data from the necessary sample size.

Simple random sampling works best if you have a lot of time and resources to conduct your study, or if you are studying a limited population that can easily be sampled.

In some cases, it might be more appropriate to use a different type of probability sampling:

• Systematic sampling involves choosing your sample based on a regular interval, rather than a fully random selection. It can also be used when you don’t have a complete list of the population.
• Stratified sampling is appropriate when you want to ensure that specific characteristics are proportionally represented in the sample. You split your population into strata (for example, divided by gender or race), and then randomly select from each of these subgroups.
• Cluster sampling is appropriate when you are unable to sample from the entire population. You divide the sample into clusters that approximately reflect the whole population, and then choose your sample from a random selection of these clusters.

Prevent plagiarism, run a free check.

There are four key steps to select a simple random sample.

Step 1: Define the population

Start by deciding on the population that you want to study.

It’s important to ensure that you have access to every individual member of the population, so that you can collect data from all those who are selected for the sample.

Step 2: Decide on the sample size

Next, you need to decide how large your sample size will be. Although larger samples provide more statistical certainty, they also cost more and require far more work.

There are several potential ways to decide upon the size of your sample, but one of the simplest involves using a formula with your desired confidence interval and confidence level , estimated size of the population you are working with, and the standard deviation of whatever you want to measure in your population.

The most common confidence interval and levels used are 0.05 and 0.95, respectively. Since you may not know the standard deviation of the population you are studying, you should choose a number high enough to account for a variety of possibilities (such as 0.5).

You can then use a sample size calculator to estimate the necessary sample size.

Step 3: Randomly select your sample

This can be done in one of two ways: the lottery or random number method.

In the lottery method , you choose the sample at random by ‘drawing from a hat’ or by using a computer program that will simulate the same action.

In the random number method , you assign every individual a number. By using a random number generator or random number tables, you then randomly pick a subset of the population. You can also use the random number function (RAND) in Microsoft Excel to generate random numbers.

Step 4: Collect data from your sample

Finally, you should collect data from your sample.

To ensure the validity of your findings, you need to make sure every individual selected actually participates in your study. If some drop out or do not participate for reasons associated with the question that you’re studying, this could bias your findings.

For example, if young participants are systematically less likely to participate in your study, your findings might not be valid due to the underrepresentation of this group.

Probability sampling means that every member of the target population has a known chance of being included in the sample.

Probability sampling methods include simple random sampling , systematic sampling , stratified sampling , and cluster sampling .

Simple random sampling is a type of probability sampling in which the researcher randomly selects a subset of participants from a population . Each member of the population has an equal chance of being selected. Data are then collected from as large a percentage as possible of this random subset.

The American Community Survey  is an example of simple random sampling . In order to collect detailed data on the population of the US, the Census Bureau officials randomly select 3.5 million households per year and use a variety of methods to convince them to fill out the survey.

If properly implemented, simple random sampling is usually the best sampling method for ensuring both internal and external validity . However, it can sometimes be impractical and expensive to implement, depending on the size of the population to be studied,

If you have a list of every member of the population and the ability to reach whichever members are selected, you can use simple random sampling.

Samples are used to make inferences about populations . Samples are easier to collect data from because they are practical, cost-effective, convenient, and manageable.

Sampling bias occurs when some members of a population are systematically more likely to be selected in a sample than others.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the ‘Cite this Scribbr article’ button to automatically add the citation to our free Reference Generator.

Thomas, L. (2023, December 18). Simple Random Sampling | Definition, Steps & Examples. Scribbr. Retrieved 9 April 2024, from https://www.scribbr.co.uk/research-methods/simple-random-sampling-method/

Is this article helpful?

Lauren Thomas

Other students also liked, sampling methods | types, techniques, & examples, a quick guide to experimental design | 5 steps & examples, what is quantitative research | definition & methods.

Iowa Reading Research Center

Technically Speaking: Why We Use Random Assignment in Reading Research

Editor’s note: This blog post is part of an ongoing series entitled  “Technically Speaking.”  In these posts, we write in a way that is understandable about very technical principles that we use in reading research. We want to improve busy practitioners’ and family members’ abilities to be good consumers of reading research and to deepen their understanding of how our research operates to provide the best information.

Making Causal Inferences with Random Assignment

In our  previous “Technically Speaking” blog post , we explained the first of two important goals when conducting a study that attempts to measure the effectiveness of a reading intervention on student outcomes:

• Be able to generalize results to a wider student population
• Establish a causal relationship between the reading intervention and changes observed in student reading

If random sampling addresses the goal of generalizing results as we wrote previously, what do researchers implement to make causal inferences about the reading intervention being studied? A key design component to facilitate this is  random assignment . Whereas random sampling helps facilitate the external validity of a study (i.e., the degree to which findings can be appropriately generalized from a sample to a population), random assignment helps establish a study’s  internal validity . This refers to the degree to which one can conclude that changes to the instruction based on the treatment and changes in students’ outcomes reflect a cause-effect relationship between the two (Shadish, Cook, & Campbell, 2002). In other words, just how sure can we be that the changes in student reading performance observed during the study are a result of the reading intervention that was implemented?

Randomly assigning students to treatment conditions means every student has an equally likely chance to end up in either the treatment/intervention group or a comparison group. A comparison group is important for establishing internal validity because it serves as a way to determine what the outcome would be in the absence of the intervention. Without a comparison group, it becomes difficult to disentangle potential effects of an intervention from factors such as general student maturation (particularly if the study extends over time), biases in how student participants were selected for the study conditions, or other uncontrollable factors. What may serve as a reasonable comparison group in a reading intervention study? Depending on the study, a comparison group could be a control group of students that do not receive any reading intervention at all (e.g., a treatment group attends summer school and a control group does not), a classroom that receives the “status quo” or usual reading instruction that students receive, or a classroom that receives an alternative reading intervention that differs from the intervention of interest in the study.

Random assignment, then, establishes that—before the intervention begins—the treatment and comparison groups are equal  on expectation . On expectation means that while student characteristics may not be necessarily the same if a handful of students from each group is selected at random, on average over all possible assignments of students to conditions, the important characteristics between treatment and control group would be similar (Shadish et al., 2002). In other words, in the long run groups will be similar to one another on average for all characteristics other than the treatment condition itself. This expected equality of groups reduces the likelihood that other factors that could affect the outcome of a study are confounded with the treatment. In other words, if groups were randomly equated prior to the intervention, then any differences we observe after the intervention is implemented could not be the result of any preexisting differences prior to intervention. By removing these potential confounding factors, random assignment reduces the plausibility of alternative explanations for any observed relationship between the reading intervention and student outcomes (Shadish et al., 2002).

Consider the alternative to random assignment—where students or teachers are assigned to groups in some planned or systematic way (e.g., only highly experienced teachers being chosen to pilot a new curriculum) or where the teachers are allowed to choose the group to which their students would belong. This leads to a  selection bias , where there are systematic differences in teacher or student qualities in the two groups that are completely independent of treatment condition. Random assignment removes, or at the very least diminishes, these selection biases.

Random assignment to groups can occur at different levels—schools, classrooms, students within classrooms, or a combination of these. For example, a study’s design may have one school in a district implement an intervention of interest while another school in the district will serve as a comparison group – ideally, the researcher would then randomly assign the intervention of interest to one of these schools, while the other school receives a different intervention. On the other hand, if different classrooms within the same school will be receiving different interventions, then random assignment of conditions to each classroom is needed. There are strengths and weaknesses for designing implementation of random assignment at each of these levels within a study.

Perhaps the biggest challenges of random assignment arise when the study design calls for students within the same classroom to receive different intervention conditions. For one, if different interventions are implemented within the same classroom, there is the concern of treatment contamination (i.e., students receiving components of the treatment when they were supposed to receive the control condition, or vice versa). On the other hand, there may be less of a teacher effect when both interventions are implemented within the same classroom by the same teacher. Another challenge is that the differences in student abilities within classrooms typically are larger than the average differences in student abilities between classrooms. This large variability within classrooms means there is a fairly high chance that students’ reading abilities will not be balanced between the treatment and comparison groups despite randomly assigning students within the class to conditions. Although there are ways statistically to try to account for these imbalances during the analysis phase of a study, there are also ways this can be addressed within the assignment process itself. Pairing students by ability and then randomly assigning one member of the pair to a condition (and thereby placing the other member in the other condition) is one method that can provide some level of group balancing while still using random assignment.

An Example of Random Assignment in Reading Research

Building on the example from our  previous “Technically Speaking” blog post  on random sampling, recall that the colored dots represent a sample of 100 students with different characteristics. We now assume that we will randomly allocate half of our sample (50 units or students) to the treatment group (Figure 1) and half of them (50 units) to the comparison group (Figure 2).

Figure 1. Treatment Group Using Simple Random Assignment

Figure 2. Comparison Group Using Simple Random Assignment

The figures above illustrate the treatment and comparison groups if simple random assignment were used. As the visual representation suggests, the important student characteristics represented by the colored dots do not appear similarly in both figures. This means that the students in these two groups may not be similar enough to compare their results, depending on how important it is for these student characteristics to be distributed evenly between the two groups. A way to address this will be discussed after this example.

Let us look at this another way. Below, we have broken down each group by the numbers. Each letter A through J represents an important student characteristic.

Treatment Group

Comparison group.

We can see above that the number of students with each characteristic in some cases differs substantially between the groups (e.g., there 6 students with characteristic J in the treatment group but only 2 students with characteristic J in the comparison group). This demonstrates that random assignment equates groups  on expectation , but perhaps not always in reality. If we are sure that these student characteristics must be evenly represented across samples, we typically rely on  block randomization . In short, this means we would create a pool or block of students by each characteristic of interest and randomly allocate students within each block. Figures 3 and 4 display the same samples as displayed in Figures 1 and 2, but in this case, block randomization was used.

Figure 3. Treatment Group Using Block Randomization

Figure 4. Comparison Group Using Block Randomization

Looking back at the breakdown of the groups by the numbers of students with each lettered characteristic, we can confirm that block randomization was more successful in equating our samples than simple random assignment (e.g., there are now 4 students with characteristic J in both the treatment and comparison groups). Remember, each letter represents an important student characteristic that should be evenly distributed between the two groups.

Random sampling and random assignment are two distinct techniques used in research. As we explained  in our previous post , random sampling aids in our ability to create representative samples from a population of interest. This post addressed how random assignment aids in creating treatment and control groups that are equated on expectation for important characteristics. Random sampling is related to the ability to generalize our study results to a wider universe of students, and random assignment is related to establishing the causal effect of a reading intervention. When used together, these techniques provide researchers sound evidence of whether or not a reading intervention has the potential to help students become more skilled readers.

Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002).  Experimental and quasi-experimental designs for generalized causal inference . Cengage Learning: Boston, MA.

• interventions
• random assignment
• technically speaking

Technically Speaking: Determining Test Effectiveness With Item Response Theory

Technically Speaking: Understanding and Quantifying the Correlation of Two Reading Measures

Technically Speaking: What is Causal Inference and Why is it Important?

Technically Speaking: Why We Use Random Sampling in Reading Research

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

Chapter 6: Experimental Research

Experimental Design

Learning Objectives

• Explain the difference between between-subjects and within-subjects experiments, list some of the pros and cons of each approach, and decide which approach to use to answer a particular research question.
• Define random assignment, distinguish it from random sampling, explain its purpose in experimental research, and use some simple strategies to implement it.
• Define what a control condition is, explain its purpose in research on treatment effectiveness, and describe some alternative types of control conditions.
• Define several types of carryover effect, give examples of each, and explain how counterbalancing helps to deal with them.

In this section, we look at some different ways to design an experiment. The primary distinction we will make is between approaches in which each participant experiences one level of the independent variable and approaches in which each participant experiences all levels of the independent variable. The former are called between-subjects experiments and the latter are called within-subjects experiments.

Between-Subjects Experiments

In a  between-subjects experiment , each participant is tested in only one condition. For example, a researcher with a sample of 100 university  students might assign half of them to write about a traumatic event and the other half write about a neutral event. Or a researcher with a sample of 60 people with severe agoraphobia (fear of open spaces) might assign 20 of them to receive each of three different treatments for that disorder. It is essential in a between-subjects experiment that the researcher assign participants to conditions so that the different groups are, on average, highly similar to each other. Those in a trauma condition and a neutral condition, for example, should include a similar proportion of men and women, and they should have similar average intelligence quotients (IQs), similar average levels of motivation, similar average numbers of health problems, and so on. This matching is a matter of controlling these extraneous participant variables across conditions so that they do not become confounding variables.

Random Assignment

The primary way that researchers accomplish this kind of control of extraneous variables across conditions is called  random assignment , which means using a random process to decide which participants are tested in which conditions. Do not confuse random assignment with random sampling. Random sampling is a method for selecting a sample from a population, and it is rarely used in psychological research. Random assignment is a method for assigning participants in a sample to the different conditions, and it is an important element of all experimental research in psychology and other fields too.

In its strictest sense, random assignment should meet two criteria. One is that each participant has an equal chance of being assigned to each condition (e.g., a 50% chance of being assigned to each of two conditions). The second is that each participant is assigned to a condition independently of other participants. Thus one way to assign participants to two conditions would be to flip a coin for each one. If the coin lands heads, the participant is assigned to Condition A, and if it lands tails, the participant is assigned to Condition B. For three conditions, one could use a computer to generate a random integer from 1 to 3 for each participant. If the integer is 1, the participant is assigned to Condition A; if it is 2, the participant is assigned to Condition B; and if it is 3, the participant is assigned to Condition C. In practice, a full sequence of conditions—one for each participant expected to be in the experiment—is usually created ahead of time, and each new participant is assigned to the next condition in the sequence as he or she is tested. When the procedure is computerized, the computer program often handles the random assignment.

One problem with coin flipping and other strict procedures for random assignment is that they are likely to result in unequal sample sizes in the different conditions. Unequal sample sizes are generally not a serious problem, and you should never throw away data you have already collected to achieve equal sample sizes. However, for a fixed number of participants, it is statistically most efficient to divide them into equal-sized groups. It is standard practice, therefore, to use a kind of modified random assignment that keeps the number of participants in each group as similar as possible. One approach is block randomization . In block randomization, all the conditions occur once in the sequence before any of them is repeated. Then they all occur again before any of them is repeated again. Within each of these “blocks,” the conditions occur in a random order. Again, the sequence of conditions is usually generated before any participants are tested, and each new participant is assigned to the next condition in the sequence.  Table 6.2  shows such a sequence for assigning nine participants to three conditions. The Research Randomizer website will generate block randomization sequences for any number of participants and conditions. Again, when the procedure is computerized, the computer program often handles the block randomization.

Random assignment is not guaranteed to control all extraneous variables across conditions. It is always possible that just by chance, the participants in one condition might turn out to be substantially older, less tired, more motivated, or less depressed on average than the participants in another condition. However, there are some reasons that this possibility is not a major concern. One is that random assignment works better than one might expect, especially for large samples. Another is that the inferential statistics that researchers use to decide whether a difference between groups reflects a difference in the population takes the “fallibility” of random assignment into account. Yet another reason is that even if random assignment does result in a confounding variable and therefore produces misleading results, this confound is likely to be detected when the experiment is replicated. The upshot is that random assignment to conditions—although not infallible in terms of controlling extraneous variables—is always considered a strength of a research design.

Treatment and Control Conditions

Between-subjects experiments are often used to determine whether a treatment works. In psychological research, a  treatment  is any intervention meant to change people’s behaviour for the better. This  intervention  includes psychotherapies and medical treatments for psychological disorders but also interventions designed to improve learning, promote conservation, reduce prejudice, and so on. To determine whether a treatment works, participants are randomly assigned to either a  treatment condition , in which they receive the treatment, or a control condition , in which they do not receive the treatment. If participants in the treatment condition end up better off than participants in the control condition—for example, they are less depressed, learn faster, conserve more, express less prejudice—then the researcher can conclude that the treatment works. In research on the effectiveness of psychotherapies and medical treatments, this type of experiment is often called a randomized clinical trial .

There are different types of control conditions. In a  no-treatment control condition , participants receive no treatment whatsoever. One problem with this approach, however, is the existence of placebo effects. A  placebo  is a simulated treatment that lacks any active ingredient or element that should make it effective, and a  placebo effect  is a positive effect of such a treatment. Many folk remedies that seem to work—such as eating chicken soup for a cold or placing soap under the bedsheets to stop nighttime leg cramps—are probably nothing more than placebos. Although placebo effects are not well understood, they are probably driven primarily by people’s expectations that they will improve. Having the expectation to improve can result in reduced stress, anxiety, and depression, which can alter perceptions and even improve immune system functioning (Price, Finniss, & Benedetti, 2008) [1] .

Placebo effects are interesting in their own right (see  Note “The Powerful Placebo” ), but they also pose a serious problem for researchers who want to determine whether a treatment works.  Figure 6.2  shows some hypothetical results in which participants in a treatment condition improved more on average than participants in a no-treatment control condition. If these conditions (the two leftmost bars in  Figure 6.2 ) were the only conditions in this experiment, however, one could not conclude that the treatment worked. It could be instead that participants in the treatment group improved more because they expected to improve, while those in the no-treatment control condition did not.

Fortunately, there are several solutions to this problem. One is to include a placebo control condition , in which participants receive a placebo that looks much like the treatment but lacks the active ingredient or element thought to be responsible for the treatment’s effectiveness. When participants in a treatment condition take a pill, for example, then those in a placebo control condition would take an identical-looking pill that lacks the active ingredient in the treatment (a “sugar pill”). In research on psychotherapy effectiveness, the placebo might involve going to a psychotherapist and talking in an unstructured way about one’s problems. The idea is that if participants in both the treatment and the placebo control groups expect to improve, then any improvement in the treatment group over and above that in the placebo control group must have been caused by the treatment and not by participants’ expectations. This  difference  is what is shown by a comparison of the two outer bars in  Figure 6.2 .

Of course, the principle of informed consent requires that participants be told that they will be assigned to either a treatment or a placebo control condition—even though they cannot be told which until the experiment ends. In many cases the participants who had been in the control condition are then offered an opportunity to have the real treatment. An alternative approach is to use a waitlist control condition , in which participants are told that they will receive the treatment but must wait until the participants in the treatment condition have already received it. This disclosure allows researchers to compare participants who have received the treatment with participants who are not currently receiving it but who still expect to improve (eventually). A final solution to the problem of placebo effects is to leave out the control condition completely and compare any new treatment with the best available alternative treatment. For example, a new treatment for simple phobia could be compared with standard exposure therapy. Because participants in both conditions receive a treatment, their expectations about improvement should be similar. This approach also makes sense because once there is an effective treatment, the interesting question about a new treatment is not simply “Does it work?” but “Does it work better than what is already available?

The Powerful Placebo

Many people are not surprised that placebos can have a positive effect on disorders that seem fundamentally psychological, including depression, anxiety, and insomnia. However, placebos can also have a positive effect on disorders that most people think of as fundamentally physiological. These include asthma, ulcers, and warts (Shapiro & Shapiro, 1999) [2] . There is even evidence that placebo surgery—also called “sham surgery”—can be as effective as actual surgery.

Medical researcher J. Bruce Moseley and his colleagues conducted a study on the effectiveness of two arthroscopic surgery procedures for osteoarthritis of the knee (Moseley et al., 2002) [3] . The control participants in this study were prepped for surgery, received a tranquilizer, and even received three small incisions in their knees. But they did not receive the actual arthroscopic surgical procedure. The surprising result was that all participants improved in terms of both knee pain and function, and the sham surgery group improved just as much as the treatment groups. According to the researchers, “This study provides strong evidence that arthroscopic lavage with or without débridement [the surgical procedures used] is not better than and appears to be equivalent to a placebo procedure in improving knee pain and self-reported function” (p. 85).

Within-Subjects Experiments

In a within-subjects experiment , each participant is tested under all conditions. Consider an experiment on the effect of a defendant’s physical attractiveness on judgments of his guilt. Again, in a between-subjects experiment, one group of participants would be shown an attractive defendant and asked to judge his guilt, and another group of participants would be shown an unattractive defendant and asked to judge his guilt. In a within-subjects experiment, however, the same group of participants would judge the guilt of both an attractive and an unattractive defendant.

The primary advantage of this approach is that it provides maximum control of extraneous participant variables. Participants in all conditions have the same mean IQ, same socioeconomic status, same number of siblings, and so on—because they are the very same people. Within-subjects experiments also make it possible to use statistical procedures that remove the effect of these extraneous participant variables on the dependent variable and therefore make the data less “noisy” and the effect of the independent variable easier to detect. We will look more closely at this idea later in the book.  However, not all experiments can use a within-subjects design nor would it be desirable to.

Carryover Effects and Counterbalancing

The primary disad vantage of within-subjects designs is that they can result in carryover effects. A  carryover effect  is an effect of being tested in one condition on participants’ behaviour in later conditions. One type of carryover effect is a  practice effect , where participants perform a task better in later conditions because they have had a chance to practice it. Another type is a fatigue effect , where participants perform a task worse in later conditions because they become tired or bored. Being tested in one condition can also change how participants perceive stimuli or interpret their task in later conditions. This  type of effect  is called a  context effect . For example, an average-looking defendant might be judged more harshly when participants have just judged an attractive defendant than when they have just judged an unattractive defendant. Within-subjects experiments also make it easier for participants to guess the hypothesis. For example, a participant who is asked to judge the guilt of an attractive defendant and then is asked to judge the guilt of an unattractive defendant is likely to guess that the hypothesis is that defendant attractiveness affects judgments of guilt. This  knowledge  could lead the participant to judge the unattractive defendant more harshly because he thinks this is what he is expected to do. Or it could make participants judge the two defendants similarly in an effort to be “fair.”

Carryover effects can be interesting in their own right. (Does the attractiveness of one person depend on the attractiveness of other people that we have seen recently?) But when they are not the focus of the research, carryover effects can be problematic. Imagine, for example, that participants judge the guilt of an attractive defendant and then judge the guilt of an unattractive defendant. If they judge the unattractive defendant more harshly, this might be because of his unattractiveness. But it could be instead that they judge him more harshly because they are becoming bored or tired. In other words, the order of the conditions is a confounding variable. The attractive condition is always the first condition and the unattractive condition the second. Thus any difference between the conditions in terms of the dependent variable could be caused by the order of the conditions and not the independent variable itself.

There is a solution to the problem of order effects, however, that can be used in many situations. It is  counterbalancing , which means testing different participants in different orders. For example, some participants would be tested in the attractive defendant condition followed by the unattractive defendant condition, and others would be tested in the unattractive condition followed by the attractive condition. With three conditions, there would be six different orders (ABC, ACB, BAC, BCA, CAB, and CBA), so some participants would be tested in each of the six orders. With counterbalancing, participants are assigned to orders randomly, using the techniques we have already discussed. Thus random assignment plays an important role in within-subjects designs just as in between-subjects designs. Here, instead of randomly assigning to conditions, they are randomly assigned to different orders of conditions. In fact, it can safely be said that if a study does not involve random assignment in one form or another, it is not an experiment.

An efficient way of counterbalancing is through a Latin square design which randomizes through having equal rows and columns. For example, if you have four treatments, you must have four versions. Like a Sudoku puzzle, no treatment can repeat in a row or column. For four versions of four treatments, the Latin square design would look like:

There are two ways to think about what counterbalancing accomplishes. One is that it controls the order of conditions so that it is no longer a confounding variable. Instead of the attractive condition always being first and the unattractive condition always being second, the attractive condition comes first for some participants and second for others. Likewise, the unattractive condition comes first for some participants and second for others. Thus any overall difference in the dependent variable between the two conditions cannot have been caused by the order of conditions. A second way to think about what counterbalancing accomplishes is that if there are carryover effects, it makes it possible to detect them. One can analyze the data separately for each order to see whether it had an effect.

When 9 is “larger” than 221

Researcher Michael Birnbaum has argued that the lack of context provided by between-subjects designs is often a bigger problem than the context effects created by within-subjects designs. To demonstrate this problem, he asked participants to rate two numbers on how large they were on a scale of 1-to-10 where 1 was “very very small” and 10 was “very very large”.  One group of participants were asked to rate the number 9 and another group was asked to rate the number 221 (Birnbaum, 1999) [4] . Participants in this between-subjects design gave the number 9 a mean rating of 5.13 and the number 221 a mean rating of 3.10. In other words, they rated 9 as larger than 221! According to Birnbaum, this difference is because participants spontaneously compared 9 with other one-digit numbers (in which case it is relatively large) and compared 221 with other three-digit numbers (in which case it is relatively small) .

Simultaneous Within-Subjects Designs

So far, we have discussed an approach to within-subjects designs in which participants are tested in one condition at a time. There is another approach, however, that is often used when participants make multiple responses in each condition. Imagine, for example, that participants judge the guilt of 10 attractive defendants and 10 unattractive defendants. Instead of having people make judgments about all 10 defendants of one type followed by all 10 defendants of the other type, the researcher could present all 20 defendants in a sequence that mixed the two types. The researcher could then compute each participant’s mean rating for each type of defendant. Or imagine an experiment designed to see whether people with social anxiety disorder remember negative adjectives (e.g., “stupid,” “incompetent”) better than positive ones (e.g., “happy,” “productive”). The researcher could have participants study a single list that includes both kinds of words and then have them try to recall as many words as possible. The researcher could then count the number of each type of word that was recalled. There are many ways to determine the order in which the stimuli are presented, but one common way is to generate a different random order for each participant.

Between-Subjects or Within-Subjects?

Almost every experiment can be conducted using either a between-subjects design or a within-subjects design. This possibility means that researchers must choose between the two approaches based on their relative merits for the particular situation.

Between-subjects experiments have the advantage of being conceptually simpler and requiring less testing time per participant. They also avoid carryover effects without the need for counterbalancing. Within-subjects experiments have the advantage of controlling extraneous participant variables, which generally reduces noise in the data and makes it easier to detect a relationship between the independent and dependent variables.

A good rule of thumb, then, is that if it is possible to conduct a within-subjects experiment (with proper counterbalancing) in the time that is available per participant—and you have no serious concerns about carryover effects—this design is probably the best option. If a within-subjects design would be difficult or impossible to carry out, then you should consider a between-subjects design instead. For example, if you were testing participants in a doctor’s waiting room or shoppers in line at a grocery store, you might not have enough time to test each participant in all conditions and therefore would opt for a between-subjects design. Or imagine you were trying to reduce people’s level of prejudice by having them interact with someone of another race. A within-subjects design with counterbalancing would require testing some participants in the treatment condition first and then in a control condition. But if the treatment works and reduces people’s level of prejudice, then they would no longer be suitable for testing in the control condition. This difficulty is true for many designs that involve a treatment meant to produce long-term change in participants’ behaviour (e.g., studies testing the effectiveness of psychotherapy). Clearly, a between-subjects design would be necessary here.

Remember also that using one type of design does not preclude using the other type in a different study. There is no reason that a researcher could not use both a between-subjects design and a within-subjects design to answer the same research question. In fact, professional researchers often take exactly this type of mixed methods approach.

Key Takeaways

• Experiments can be conducted using either between-subjects or within-subjects designs. Deciding which to use in a particular situation requires careful consideration of the pros and cons of each approach.
• Random assignment to conditions in between-subjects experiments or to orders of conditions in within-subjects experiments is a fundamental element of experimental research. Its purpose is to control extraneous variables so that they do not become confounding variables.
• Experimental research on the effectiveness of a treatment requires both a treatment condition and a control condition, which can be a no-treatment control condition, a placebo control condition, or a waitlist control condition. Experimental treatments can also be compared with the best available alternative.
• You want to test the relative effectiveness of two training programs for running a marathon.
• Using photographs of people as stimuli, you want to see if smiling people are perceived as more intelligent than people who are not smiling.
• In a field experiment, you want to see if the way a panhandler is dressed (neatly vs. sloppily) affects whether or not passersby give him any money.
• You want to see if concrete nouns (e.g.,  dog ) are recalled better than abstract nouns (e.g.,  truth ).
• Discussion: Imagine that an experiment shows that participants who receive psychodynamic therapy for a dog phobia improve more than participants in a no-treatment control group. Explain a fundamental problem with this research design and at least two ways that it might be corrected.
• Price, D. D., Finniss, D. G., & Benedetti, F. (2008). A comprehensive review of the placebo effect: Recent advances and current thought. Annual Review of Psychology, 59 , 565–590. ↵
• Shapiro, A. K., & Shapiro, E. (1999). The powerful placebo: From ancient priest to modern physician . Baltimore, MD: Johns Hopkins University Press. ↵
• Moseley, J. B., O’Malley, K., Petersen, N. J., Menke, T. J., Brody, B. A., Kuykendall, D. H., … Wray, N. P. (2002). A controlled trial of arthroscopic surgery for osteoarthritis of the knee. The New England Journal of Medicine, 347 , 81–88. ↵
• Birnbaum, M.H. (1999). How to show that 9>221: Collect judgments in a between-subjects design. Psychological Methods, 4(3), 243-249. ↵

An experiment in which each participant is only tested in one condition.

A method of controlling extraneous variables across conditions by using a random process to decide which participants will be tested in the different conditions.

All the conditions of an experiment occur once in the sequence before any of them is repeated.

Any intervention meant to change people’s behaviour for the better.

A condition in a study where participants receive treatment.

A condition in a study that the other condition is compared to. This group does not receive the treatment or intervention that the other conditions do.

A type of experiment to research the effectiveness of psychotherapies and medical treatments.

A type of control condition in which participants receive no treatment.

A simulated treatment that lacks any active ingredient or element that should make it effective.

A positive effect of a treatment that lacks any active ingredient or element to make it effective.

Participants receive a placebo that looks like the treatment but lacks the active ingredient or element thought to be responsible for the treatment’s effectiveness.

Participants are told that they will receive the treatment but must wait until the participants in the treatment condition have already received it.

Each participant is tested under all conditions.

An effect of being tested in one condition on participants’ behaviour in later conditions.

Participants perform a task better in later conditions because they have had a chance to practice it.

Participants perform a task worse in later conditions because they become tired or bored.

Being tested in one condition can also change how participants perceive stimuli or interpret their task in later conditions.

Testing different participants in different orders.

Research Methods in Psychology - 2nd Canadian Edition Copyright © 2015 by Paul C. Price, Rajiv Jhangiani, & I-Chant A. Chiang is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License , except where otherwise noted.

Share This Book