types of non experimental research design ppt

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

7.1 Overview of Nonexperimental Research

Learning objectives.

• Define nonexperimental research, distinguish it clearly from experimental research, and give several examples.
• Explain when a researcher might choose to conduct nonexperimental research as opposed to experimental research.

What Is Nonexperimental Research?

Nonexperimental research is research that lacks the manipulation of an independent variable, random assignment of participants to conditions or orders of conditions, or both.

In a sense, it is unfair to define this large and diverse set of approaches collectively by what they are not . But doing so reflects the fact that most researchers in psychology consider the distinction between experimental and nonexperimental research to be an extremely important one. This is because while experimental research can provide strong evidence that changes in an independent variable cause differences in a dependent variable, nonexperimental research generally cannot. As we will see, however, this does not mean that nonexperimental research is less important than experimental research or inferior to it in any general sense.

When to Use Nonexperimental Research

As we saw in Chapter 6 “Experimental Research” , experimental research is appropriate when the researcher has a specific research question or hypothesis about a causal relationship between two variables—and it is possible, feasible, and ethical to manipulate the independent variable and randomly assign participants to conditions or to orders of conditions. It stands to reason, therefore, that nonexperimental research is appropriate—even necessary—when these conditions are not met. There are many ways in which this can be the case.

• The research question or hypothesis can be about a single variable rather than a statistical relationship between two variables (e.g., How accurate are people’s first impressions?).
• The research question can be about a noncausal statistical relationship between variables (e.g., Is there a correlation between verbal intelligence and mathematical intelligence?).
• The research question can be about a causal relationship, but the independent variable cannot be manipulated or participants cannot be randomly assigned to conditions or orders of conditions (e.g., Does damage to a person’s hippocampus impair the formation of long-term memory traces?).
• The research question can be broad and exploratory, or it can be about what it is like to have a particular experience (e.g., What is it like to be a working mother diagnosed with depression?).

Again, the choice between the experimental and nonexperimental approaches is generally dictated by the nature of the research question. If it is about a causal relationship and involves an independent variable that can be manipulated, the experimental approach is typically preferred. Otherwise, the nonexperimental approach is preferred. But the two approaches can also be used to address the same research question in complementary ways. For example, nonexperimental studies establishing that there is a relationship between watching violent television and aggressive behavior have been complemented by experimental studies confirming that the relationship is a causal one (Bushman & Huesmann, 2001). Similarly, after his original study, Milgram conducted experiments to explore the factors that affect obedience. He manipulated several independent variables, such as the distance between the experimenter and the participant, the participant and the confederate, and the location of the study (Milgram, 1974).

Types of Nonexperimental Research

Nonexperimental research falls into three broad categories: single-variable research, correlational and quasi-experimental research, and qualitative research. First, research can be nonexperimental because it focuses on a single variable rather than a statistical relationship between two variables. Although there is no widely shared term for this kind of research, we will call it single-variable research . Milgram’s original obedience study was nonexperimental in this way. He was primarily interested in one variable—the extent to which participants obeyed the researcher when he told them to shock the confederate—and he observed all participants performing the same task under the same conditions. The study by Loftus and Pickrell described at the beginning of this chapter is also a good example of single-variable research. The variable was whether participants “remembered” having experienced mildly traumatic childhood events (e.g., getting lost in a shopping mall) that they had not actually experienced but that the research asked them about repeatedly. In this particular study, nearly a third of the participants “remembered” at least one event. (As with Milgram’s original study, this study inspired several later experiments on the factors that affect false memories.)

As these examples make clear, single-variable research can answer interesting and important questions. What it cannot do, however, is answer questions about statistical relationships between variables. This is a point that beginning researchers sometimes miss. Imagine, for example, a group of research methods students interested in the relationship between children’s being the victim of bullying and the children’s self-esteem. The first thing that is likely to occur to these researchers is to obtain a sample of middle-school students who have been bullied and then to measure their self-esteem. But this would be a single-variable study with self-esteem as the only variable. Although it would tell the researchers something about the self-esteem of children who have been bullied, it would not tell them what they really want to know, which is how the self-esteem of children who have been bullied compares with the self-esteem of children who have not. Is it lower? Is it the same? Could it even be higher? To answer this question, their sample would also have to include middle-school students who have not been bullied.

Research can also be nonexperimental because it focuses on a statistical relationship between two variables but does not include the manipulation of an independent variable, random assignment of participants to conditions or orders of conditions, or both. This kind of research takes two basic forms: correlational research and quasi-experimental research. In correlational research , the researcher measures the two variables of interest with little or no attempt to control extraneous variables and then assesses the relationship between them. A research methods student who finds out whether each of several middle-school students has been bullied and then measures each student’s self-esteem is conducting correlational research. In quasi-experimental research , the researcher manipulates an independent variable but does not randomly assign participants to conditions or orders of conditions. For example, a researcher might start an antibullying program (a kind of treatment) at one school and compare the incidence of bullying at that school with the incidence at a similar school that has no antibullying program.

The final way in which research can be nonexperimental is that it can be qualitative. The types of research we have discussed so far are all quantitative, referring to the fact that the data consist of numbers that are analyzed using statistical techniques. In qualitative research , the data are usually nonnumerical and are analyzed using nonstatistical techniques. Rosenhan’s study of the experience of people in a psychiatric ward was primarily qualitative. The data were the notes taken by the “pseudopatients”—the people pretending to have heard voices—along with their hospital records. Rosenhan’s analysis consists mainly of a written description of the experiences of the pseudopatients, supported by several concrete examples. To illustrate the hospital staff’s tendency to “depersonalize” their patients, he noted, “Upon being admitted, I and other pseudopatients took the initial physical examinations in a semipublic room, where staff members went about their own business as if we were not there” (Rosenhan, 1973, p. 256).

Internal Validity Revisited

Recall that internal validity is the extent to which the design of a study supports the conclusion that changes in the independent variable caused any observed differences in the dependent variable. Figure 7.1 shows how experimental, quasi-experimental, and correlational research vary in terms of internal validity. Experimental research tends to be highest because it addresses the directionality and third-variable problems through manipulation and the control of extraneous variables through random assignment. If the average score on the dependent variable in an experiment differs across conditions, it is quite likely that the independent variable is responsible for that difference. Correlational research is lowest because it fails to address either problem. If the average score on the dependent variable differs across levels of the independent variable, it could be that the independent variable is responsible, but there are other interpretations. In some situations, the direction of causality could be reversed. In others, there could be a third variable that is causing differences in both the independent and dependent variables. Quasi-experimental research is in the middle because the manipulation of the independent variable addresses some problems, but the lack of random assignment and experimental control fails to address others. Imagine, for example, that a researcher finds two similar schools, starts an antibullying program in one, and then finds fewer bullying incidents in that “treatment school” than in the “control school.” There is no directionality problem because clearly the number of bullying incidents did not determine which school got the program. However, the lack of random assignment of children to schools could still mean that students in the treatment school differed from students in the control school in some other way that could explain the difference in bullying.

Experiments are generally high in internal validity, quasi-experiments lower, and correlational studies lower still.

Notice also in Figure 7.1 that there is some overlap in the internal validity of experiments, quasi-experiments, and correlational studies. For example, a poorly designed experiment that includes many confounding variables can be lower in internal validity than a well designed quasi-experiment with no obvious confounding variables.

Key Takeaways

• Nonexperimental research is research that lacks the manipulation of an independent variable, control of extraneous variables through random assignment, or both.
• There are three broad types of nonexperimental research. Single-variable research focuses on a single variable rather than a relationship between variables. Correlational and quasi-experimental research focus on a statistical relationship but lack manipulation or random assignment. Qualitative research focuses on broader research questions, typically involves collecting large amounts of data from a small number of participants, and analyzes the data nonstatistically.
• In general, experimental research is high in internal validity, correlational research is low in internal validity, and quasi-experimental research is in between.

Discussion: For each of the following studies, decide which type of research design it is and explain why.

• A researcher conducts detailed interviews with unmarried teenage fathers to learn about how they feel and what they think about their role as fathers and summarizes their feelings in a written narrative.
• A researcher measures the impulsivity of a large sample of drivers and looks at the statistical relationship between this variable and the number of traffic tickets the drivers have received.
• A researcher randomly assigns patients with low back pain either to a treatment involving hypnosis or to a treatment involving exercise. She then measures their level of low back pain after 3 months.
• A college instructor gives weekly quizzes to students in one section of his course but no weekly quizzes to students in another section to see whether this has an effect on their test performance.

Bushman, B. J., & Huesmann, L. R. (2001). Effects of televised violence on aggression. In D. Singer & J. Singer (Eds.), Handbook of children and the media (pp. 223–254). Thousand Oaks, CA: Sage.

Milgram, S. (1974). Obedience to authority: An experimental view . New York, NY: Harper & Row.

Rosenhan, D. L. (1973). On being sane in insane places. Science, 179 , 250–258.

Research Methods in Psychology Copyright © 2016 by University of Minnesota is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License , except where otherwise noted.

6.1 Overview of Non-Experimental Research

Learning objectives.

• Define non-experimental research, distinguish it clearly from experimental research, and give several examples.
• Explain when a researcher might choose to conduct non-experimental research as opposed to experimental research.

What Is Non-Experimental Research?

Non-experimental research  is research that lacks the manipulation of an independent variable. Rather than manipulating an independent variable, researchers conducting non-experimental research simply measure variables as they naturally occur (in the lab or real world).

Most researchers in psychology consider the distinction between experimental and non-experimental research to be an extremely important one. This is because although experimental research can provide strong evidence that changes in an independent variable cause differences in a dependent variable, non-experimental research generally cannot. As we will see, however, this inability to make causal conclusions does not mean that non-experimental research is less important than experimental research.

When to Use Non-Experimental Research

As we saw in the last chapter , experimental research is appropriate when the researcher has a specific research question or hypothesis about a causal relationship between two variables—and it is possible, feasible, and ethical to manipulate the independent variable. It stands to reason, therefore, that non-experimental research is appropriate—even necessary—when these conditions are not met. There are many times in which non-experimental research is preferred, including when:

• the research question or hypothesis relates to a single variable rather than a statistical relationship between two variables (e.g., How accurate are people’s first impressions?).
• the research question pertains to a non-causal statistical relationship between variables (e.g., is there a correlation between verbal intelligence and mathematical intelligence?).
• the research question is about a causal relationship, but the independent variable cannot be manipulated or participants cannot be randomly assigned to conditions or orders of conditions for practical or ethical reasons (e.g., does damage to a person’s hippocampus impair the formation of long-term memory traces?).
• the research question is broad and exploratory, or is about what it is like to have a particular experience (e.g., what is it like to be a working mother diagnosed with depression?).

Again, the choice between the experimental and non-experimental approaches is generally dictated by the nature of the research question. Recall the three goals of science are to describe, to predict, and to explain. If the goal is to explain and the research question pertains to causal relationships, then the experimental approach is typically preferred. If the goal is to describe or to predict, a non-experimental approach will suffice. But the two approaches can also be used to address the same research question in complementary ways. For example, Similarly, after his original study, Milgram conducted experiments to explore the factors that affect obedience. He manipulated several independent variables, such as the distance between the experimenter and the participant, the participant and the confederate, and the location of the study (Milgram, 1974) [1] .

Types of Non-Experimental Research

Non-experimental research falls into three broad categories: cross-sectional research, correlational research, and observational research. 

First, cross-sectional research  involves comparing two or more pre-existing groups of people. What makes this approach non-experimental is that there is no manipulation of an independent variable and no random assignment of participants to groups. Imagine, for example, that a researcher administers the Rosenberg Self-Esteem Scale to 50 American college students and 50 Japanese college students. Although this “feels” like a between-subjects experiment, it is a cross-sectional study because the researcher did not manipulate the students’ nationalities. As another example, if we wanted to compare the memory test performance of a group of cannabis users with a group of non-users, this would be considered a cross-sectional study because for ethical and practical reasons we would not be able to randomly assign participants to the cannabis user and non-user groups. Rather we would need to compare these pre-existing groups which could introduce a selection bias (the groups may differ in other ways that affect their responses on the dependent variable). For instance, cannabis users are more likely to use more alcohol and other drugs and these differences may account for differences in the dependent variable across groups, rather than cannabis use per se.

Cross-sectional designs are commonly used by developmental psychologists who study aging and by researchers interested in sex differences. Using this design, developmental psychologists compare groups of people of different ages (e.g., young adults spanning from 18-25 years of age versus older adults spanning 60-75 years of age) on various dependent variables (e.g., memory, depression, life satisfaction). Of course, the primary limitation of using this design to study the effects of aging is that differences between the groups other than age may account for differences in the dependent variable. For instance, differences between the groups may reflect the generation that people come from (a cohort effect) rather than a direct effect of age. For this reason, longitudinal studies in which one group of people is followed as they age offer a superior means of studying the effects of aging. Once again, cross-sectional designs are also commonly used to study sex differences. Since researchers cannot practically or ethically manipulate the sex of their participants they must rely on cross-sectional designs to compare groups of men and women on different outcomes (e.g., verbal ability, substance use, depression). Using these designs researchers have discovered that men are more likely than women to suffer from substance abuse problems while women are more likely than men to suffer from depression. But, using this design it is unclear what is causing these differences. So, using this design it is unclear whether these differences are due to environmental factors like socialization or biological factors like hormones?

When researchers use a participant characteristic to create groups (nationality, cannabis use, age, sex), the independent variable is usually referred to as an experimenter-selected independent variable (as opposed to the experimenter-manipulated independent variables used in experimental research). Figure 6.1 shows data from a hypothetical study on the relationship between whether people make a daily list of things to do (a “to-do list”) and stress. Notice that it is unclear whether this is an experiment or a cross-sectional study because it is unclear whether the independent variable was manipulated by the researcher or simply selected by the researcher. If the researcher randomly assigned some participants to make daily to-do lists and others not to, then the independent variable was experimenter-manipulated and it is a true experiment. If the researcher simply asked participants whether they made daily to-do lists or not, then the independent variable it is experimenter-selected and the study is cross-sectional. The distinction is important because if the study was an experiment, then it could be concluded that making the daily to-do lists reduced participants’ stress. But if it was a cross-sectional study, it could only be concluded that these variables are statistically related. Perhaps being stressed has a negative effect on people’s ability to plan ahead. Or perhaps people who are more conscientious are more likely to make to-do lists and less likely to be stressed. The crucial point is that what defines a study as experimental or cross-sectional l is not the variables being studied, nor whether the variables are quantitative or categorical, nor the type of graph or statistics used to analyze the data. It is how the study is conducted.

Figure 6.1  Results of a Hypothetical Study on Whether People Who Make Daily To-Do Lists Experience Less Stress Than People Who Do Not Make Such Lists

Second, the most common type of non-experimental research conducted in Psychology is correlational research. Correlational research is considered non-experimental because it focuses on the statistical relationship between two variables but does not include the manipulation of an independent variable.  More specifically, in correlational research , the researcher measures two continuous variables with little or no attempt to control extraneous variables and then assesses the relationship between them. As an example, a researcher interested in the relationship between self-esteem and school achievement could collect data on students’ self-esteem and their GPAs to see if the two variables are statistically related. Correlational research is very similar to cross-sectional research, and sometimes these terms are used interchangeably. The distinction that will be made in this book is that, rather than comparing two or more pre-existing groups of people as is done with cross-sectional research, correlational research involves correlating two continuous variables (groups are not formed and compared).

Third,   observational research  is non-experimental because it focuses on making observations of behavior in a natural or laboratory setting without manipulating anything. Milgram’s original obedience study was non-experimental in this way. He was primarily interested in the extent to which participants obeyed the researcher when he told them to shock the confederate and he observed all participants performing the same task under the same conditions. The study by Loftus and Pickrell described at the beginning of this chapter is also a good example of observational research. The variable was whether participants “remembered” having experienced mildly traumatic childhood events (e.g., getting lost in a shopping mall) that they had not actually experienced but that the researchers asked them about repeatedly. In this particular study, nearly a third of the participants “remembered” at least one event. (As with Milgram’s original study, this study inspired several later experiments on the factors that affect false memories.

The types of research we have discussed so far are all quantitative, referring to the fact that the data consist of numbers that are analyzed using statistical techniques. But as you will learn in this chapter, many observational research studies are more qualitative in nature. In  qualitative research , the data are usually nonnumerical and therefore cannot be analyzed using statistical techniques. Rosenhan’s observational study of the experience of people in a psychiatric ward was primarily qualitative. The data were the notes taken by the “pseudopatients”—the people pretending to have heard voices—along with their hospital records. Rosenhan’s analysis consists mainly of a written description of the experiences of the pseudopatients, supported by several concrete examples. To illustrate the hospital staff’s tendency to “depersonalize” their patients, he noted, “Upon being admitted, I and other pseudopatients took the initial physical examinations in a semi-public room, where staff members went about their own business as if we were not there” (Rosenhan, 1973, p. 256) [2] . Qualitative data has a separate set of analysis tools depending on the research question. For example, thematic analysis would focus on themes that emerge in the data or conversation analysis would focus on the way the words were said in an interview or focus group.

Internal Validity Revisited

Recall that internal validity is the extent to which the design of a study supports the conclusion that changes in the independent variable caused any observed differences in the dependent variable.  Figure 6.2  shows how experimental, quasi-experimental, and non-experimental (correlational) research vary in terms of internal validity. Experimental research tends to be highest in internal validity because the use of manipulation (of the independent variable) and control (of extraneous variables) help to rule out alternative explanations for the observed relationships. If the average score on the dependent variable in an experiment differs across conditions, it is quite likely that the independent variable is responsible for that difference. Non-experimental (correlational) research is lowest in internal validity because these designs fail to use manipulation or control. Quasi-experimental research (which will be described in more detail in a subsequent chapter) is in the middle because it contains some, but not all, of the features of a true experiment. For instance, it may fail to use random assignment to assign participants to groups or fail to use counterbalancing to control for potential order effects. Imagine, for example, that a researcher finds two similar schools, starts an anti-bullying program in one, and then finds fewer bullying incidents in that “treatment school” than in the “control school.” While a comparison is being made with a control condition, the lack of random assignment of children to schools could still mean that students in the treatment school differed from students in the control school in some other way that could explain the difference in bullying (e.g., there may be a selection effect).

Figure 6.2 Internal Validity of Correlation, Quasi-Experimental, and Experimental Studies. Experiments are generally high in internal validity, quasi-experiments lower, and correlation studies lower still.

Notice also in  Figure 6.2  that there is some overlap in the internal validity of experiments, quasi-experiments, and correlational studies. For example, a poorly designed experiment that includes many confounding variables can be lower in internal validity than a well-designed quasi-experiment with no obvious confounding variables. Internal validity is also only one of several validities that one might consider, as noted in Chapter 5.

Key Takeaways

• Non-experimental research is research that lacks the manipulation of an independent variable.
• There are two broad types of non-experimental research. Correlational research that focuses on statistical relationships between variables that are measured but not manipulated, and observational research in which participants are observed and their behavior is recorded without the researcher interfering or manipulating any variables.
• In general, experimental research is high in internal validity, correlational research is low in internal validity, and quasi-experimental research is in between.
• A researcher conducts detailed interviews with unmarried teenage fathers to learn about how they feel and what they think about their role as fathers and summarizes their feelings in a written narrative.
• A researcher measures the impulsivity of a large sample of drivers and looks at the statistical relationship between this variable and the number of traffic tickets the drivers have received.
• A researcher randomly assigns patients with low back pain either to a treatment involving hypnosis or to a treatment involving exercise. She then measures their level of low back pain after 3 months.
• A college instructor gives weekly quizzes to students in one section of his course but no weekly quizzes to students in another section to see whether this has an effect on their test performance.
• Milgram, S. (1974). Obedience to authority: An experimental view . New York, NY: Harper & Row. ↵
• Rosenhan, D. L. (1973). On being sane in insane places. Science, 179 , 250–258. ↵

Share This Book

• Increase Font Size

• My presentations

Auth with social network:

Download presentation

We think you have liked this presentation. If you wish to download it, please recommend it to your friends in any social system. Share buttons are a little bit lower. Thank you!

Presentation is loading. Please wait.

Myers chapter 1 (B): Non-Experimental Research Designs

Published by Adrian Shaw Modified over 5 years ago

Similar presentations

Presentation on theme: "Myers chapter 1 (B): Non-Experimental Research Designs"— Presentation transcript:

Chapter 2 Psychological Research Methods and Statistics

Myers’ PSYCHOLOGY (7th Ed)

Myers’ PSYCHOLOGY (7th Ed) Chapter 1 Thinking Critically with Psychological Science James A. McCubbin, PhD Clemson University Worth Publishers.

Critical Thinking.

Research Strategies: How Psychologists Ask and Answer Questions Module 3 Psychology 7e in Modules.

Myers EXPLORING PSYCHOLOGY (6th Edition in Modules)

Chapter 2.1: What is Research? Mr. McCormick Psychology.

Unit 2 (B): Non-Experimental Research Designs Mr. McCormick A.P. Psychology.

Research Strategies, Part 2

Chapter 2 Psychology’s Scientific Method

Research Methods Unit II.

OPERATIONAL DEFINITION a statement of the procedures used to define research variables.

OBJECTIVE 7. STUDENTS WILL DEMONSTRATE UNDERSTANDING BY EXPLAINING QUANTITATIVE RESEARCH METHODS.

Unit 1 Chapter 2.1 What is Research? Psychology 1 st and 6 th Mr. Young.

What is Research? Chapter 2 Section 1.

Chapter 1 Research Strategies: How Psychologists Ask and Answer Questions.

Myers EXPLORING PSYCHOLOGY (6th Edition in Modules) Module 2 Research Strategies: How Psychologists Ask and Answer Questions James A. McCubbin, PhD Clemson.

DESCRIPTIVE METHODS Methods that yield descriptions of behavior but not necessarily causal explanations.

Research in Psychology Chapter Two 8-10% of Exam AP Psychology.

Definition Slides Unit 2: Scientific Research Methods.

About project

© 2024 SlidePlayer.com Inc. All rights reserved.

• school Campus Bookshelves
• menu_book Bookshelves
• perm_media Learning Objects
• login Login
• how_to_reg Request Instructor Account
• hub Instructor Commons

Margin Size

• Download Page (PDF)
• Download Full Book (PDF)
• Periodic Table
• Physics Constants
• Scientific Calculator
• Reference & Cite
• Tools expand_more
• Readability

selected template will load here

This action is not available.

2.5: Experimental and Non-experimental Research

• Last updated
• Save as PDF
• Page ID 3948

• Danielle Navarro
• University of New South Wales

\( \newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)

\( \newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}} \)

\( \newcommand{\id}{\mathrm{id}}\) \( \newcommand{\Span}{\mathrm{span}}\)

( \newcommand{\kernel}{\mathrm{null}\,}\) \( \newcommand{\range}{\mathrm{range}\,}\)

\( \newcommand{\RealPart}{\mathrm{Re}}\) \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\)

\( \newcommand{\Argument}{\mathrm{Arg}}\) \( \newcommand{\norm}[1]{\| #1 \|}\)

\( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\)

\( \newcommand{\Span}{\mathrm{span}}\)

\( \newcommand{\id}{\mathrm{id}}\)

\( \newcommand{\kernel}{\mathrm{null}\,}\)

\( \newcommand{\range}{\mathrm{range}\,}\)

\( \newcommand{\RealPart}{\mathrm{Re}}\)

\( \newcommand{\ImaginaryPart}{\mathrm{Im}}\)

\( \newcommand{\Argument}{\mathrm{Arg}}\)

\( \newcommand{\norm}[1]{\| #1 \|}\)

\( \newcommand{\Span}{\mathrm{span}}\) \( \newcommand{\AA}{\unicode[.8,0]{x212B}}\)

\( \newcommand{\vectorA}[1]{\vec{#1}}      % arrow\)

\( \newcommand{\vectorAt}[1]{\vec{\text{#1}}}      % arrow\)

\( \newcommand{\vectorB}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)

\( \newcommand{\vectorC}[1]{\textbf{#1}} \)

\( \newcommand{\vectorD}[1]{\overrightarrow{#1}} \)

\( \newcommand{\vectorDt}[1]{\overrightarrow{\text{#1}}} \)

\( \newcommand{\vectE}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash{\mathbf {#1}}}} \)

One of the big distinctions that you should be aware of is the distinction between “experimental research” and “non-experimental research”. When we make this distinction, what we’re really talking about is the degree of control that the researcher exercises over the people and events in the study.

Experimental research

The key features of experimental research is that the researcher controls all aspects of the study, especially what participants experience during the study. In particular, the researcher manipulates or varies the predictor variables (IVs), and then allows the outcome variable (DV) to vary naturally. The idea here is to deliberately vary the predictors (IVs) to see if they have any causal effects on the outcomes. Moreover, in order to ensure that there’s no chance that something other than the predictor variables is causing the outcomes, everything else is kept constant or is in some other way “balanced” to ensure that they have no effect on the results. In practice, it’s almost impossible to think of everything else that might have an influence on the outcome of an experiment, much less keep it constant. The standard solution to this is randomisation: that is, we randomly assign people to different groups, and then give each group a different treatment (i.e., assign them different values of the predictor variables). We’ll talk more about randomisation later in this course, but for now, it’s enough to say that what randomisation does is minimise (but not eliminate) the chances that there are any systematic difference between groups.

Let’s consider a very simple, completely unrealistic and grossly unethical example. Suppose you wanted to find out if smoking causes lung cancer. One way to do this would be to find people who smoke and people who don’t smoke, and look to see if smokers have a higher rate of lung cancer. This is not a proper experiment, since the researcher doesn’t have a lot of control over who is and isn’t a smoker. And this really matters: for instance, it might be that people who choose to smoke cigarettes also tend to have poor diets, or maybe they tend to work in asbestos mines, or whatever. The point here is that the groups (smokers and non-smokers) actually differ on lots of things, not just smoking. So it might be that the higher incidence of lung cancer among smokers is caused by something else, not by smoking per se. In technical terms, these other things (e.g. diet) are called “confounds”, and we’ll talk about those in just a moment.

In the meantime, let’s now consider what a proper experiment might look like. Recall that our concern was that smokers and non-smokers might differ in lots of ways. The solution, as long as you have no ethics, is to control who smokes and who doesn’t. Specifically, if we randomly divide participants into two groups, and force half of them to become smokers, then it’s very unlikely that the groups will differ in any respect other than the fact that half of them smoke. That way, if our smoking group gets cancer at a higher rate than the non-smoking group, then we can feel pretty confident that (a) smoking does cause cancer and (b) we’re murderers.

Non-experimental research

Non-experimental research is a broad term that covers “any study in which the researcher doesn’t have quite as much control as they do in an experiment”. Obviously, control is something that scientists like to have, but as the previous example illustrates, there are lots of situations in which you can’t or shouldn’t try to obtain that control. Since it’s grossly unethical (and almost certainly criminal) to force people to smoke in order to find out if they get cancer, this is a good example of a situation in which you really shouldn’t try to obtain experimental control. But there are other reasons too. Even leaving aside the ethical issues, our “smoking experiment” does have a few other issues. For instance, when I suggested that we “force” half of the people to become smokers, I must have been talking about starting with a sample of non-smokers, and then forcing them to become smokers. While this sounds like the kind of solid, evil experimental design that a mad scientist would love, it might not be a very sound way of investigating the effect in the real world. For instance, suppose that smoking only causes lung cancer when people have poor diets, and suppose also that people who normally smoke do tend to have poor diets. However, since the “smokers” in our experiment aren’t “natural” smokers (i.e., we forced non-smokers to become smokers; they didn’t take on all of the other normal, real life characteristics that smokers might tend to possess) they probably have better diets. As such, in this silly example they wouldn’t get lung cancer, and our experiment will fail, because it violates the structure of the “natural” world (the technical name for this is an “artifactual” result; see later).

One distinction worth making between two types of non-experimental research is the difference be- tween quasi-experimental research and case studies. The example I discussed earlier – in which we wanted to examine incidence of lung cancer among smokers and non-smokers, without trying to control who smokes and who doesn’t – is a quasi-experimental design. That is, it’s the same as an experiment, but we don’t control the predictors (IVs). We can still use statistics to analyse the results, it’s just that we have to be a lot more careful.

The alternative approach, case studies, aims to provide a very detailed description of one or a few instances. In general, you can’t use statistics to analyse the results of case studies, and it’s usually very hard to draw any general conclusions about “people in general” from a few isolated examples. However, case studies are very useful in some situations. Firstly, there are situations where you don’t have any alternative: neuropsychology has this issue a lot. Sometimes, you just can’t find a lot of people with brain damage in a specific area, so the only thing you can do is describe those cases that you do have in as much detail and with as much care as you can. However, there’s also some genuine advantages to case studies: because you don’t have as many people to study, you have the ability to invest lots of time and effort trying to understand the specific factors at play in each case. This is a very valuable thing to do. As a consequence, case studies can complement the more statistically-oriented approaches that you see in experimental and quasi-experimental designs. We won’t talk much about case studies in these lectures, but they are nevertheless very valuable tools!

• Preferences

Non-Experimental designs: Surveys - PowerPoint PPT Presentation

Something went wrong! Please try again and reload the page.

Non-Experimental designs: Surveys

Non-experimental designs: surveys psych 231: research methods in psychology – powerpoint ppt presentation.

• Psych 231 Research Methods in Psychology
• Sometimes you just cant perform a fully controlled experiment
• Because of the issue of interest
• Limited resources (not enough subjects, observations are too costly, etc).
• Correlational studies
• Quasi-Experiments
• Developmental designs
• Small-N designs
• This does NOT imply that they are bad designs
• Just remember the advantages and disadvantages of each
• What are they?
• Questionnaires and interviews that ask people to provide information about themselves
• Why conduct them?
• Best way to collect some kinds of information
• Descriptive, behavioral, and preferential
• e.g., demographic information, recreational behavior, and attitudes
• To compliment experimental work
• Good/common first step, can collect a lot of data about a lot of variables
• Do not have to directly observe behaviors
• One can investigate internal events (for example, attitudes opinions)
• Can generalize about an entire population based on relatively small samples of individuals
• Large amounts of data can be collected quickly with relatively little cost (effort, time, etc.)
• But theyre often not as cheap as you may think
• Disadvantages
• Correlational causal claims shouldnt be made
• Note surveys are sometimes used as part of an experimental design, which may allow testing of causal claims
• Non-response bias
• Why doesnt everybody respond?
• Does response rate interact with variables of interest?
• Large data sets are sometimes difficult to analyze
• Self-reports may not be truthful
• Response set - tendency to respond from a particular perspective
• Social desirability bias (e.g., how a moral person would answer)
• Stage 1) Identify the focus of the study and select your research method
• What are the objectives of the research?
• Is a survey method the best approach?
• What kind of survey should be used?
• Group administration
• Mail surveys
• Internet surveys
• Telephone surveys
• Face-to-face interviews
• Focus group interviews
• Stage 2) Determining the research schedule and budget
• Stage 3) Establishing an information base
• Find out whats been done, whats known
• E.g., Find other related surveys
• Stage 4) Identify the sampling frame
• The actual population that the sample is drawn from (as opposed to the ideal population)
• Think of it as operationalizing the conceptual level population
• Be aware of potential coverage error when the sampling doesnt lead to a good representativeness
• Stage 5) Determining the sample method and sampling size
• Review Probability and Non-Probability methods
• Voluntary response method
• Importance of sample size
• A kind of convenience sampling methods commonly used
• Problem Typically only individuals with strong opinions respond, so the results are often extremely biased
• Sampling error - how is the sample different from the population?
• Confidence intervals
• An estimate of the mean or percentage of the population, based on the sample data
• John Doe has 55 of the vote, with a margin of error 3
• Margin of error (that 3 part)
• The larger your sample size, the smaller your margin of error will be.
• Which would you be more likely to believe
• We asked 10 people
• We asked 1000 people
• Response rate
• What proportion of the sample actually responded to the survey?
• Hidden costs here - what can you do to increase response rates
• Non-response error (bias)
• Is there something special about the data that youre missing (From the people who didnt respond)?
• Stage 6) Designing the survey instrument
• Question construction How the questions are written is very important
• Clearly identify the research objectives
• Do your questions really target those research objectives (think Internal and External Validity)?
• Take care wording of the questions
• Keep it simple, dont ask two things at once, avoid loaded or biased questions, etc.
• How should questions be answered (question type)?
• Was the FDC negligent by ignoring the
• warnings about Vioxx during testing
• and approving it for sale?
• Do you favor eliminating the wasteful
• excess in the public school budget?
• If the FDC knew that Vioxx caused serious side effects during testing,
• what should it have done?
• Ban it from ever being sold
• Require more testing before approving it
• Do you favor reducing the public school budget?
• Should senior citizens be given more
• money for recreation centers and
• food assistance programs?
• Should senior citizens be given more money for recreation centers?
• Should senior citizens be given more money for food assistance programs?
• Are you against same sex marriage
• and in favor of a constitutional
• amendment to ban it?
• What is your view on same sex marriage?
• I think marriage is a matter of personal choice
• Im against it but dont want a constitutional amendment
• I want a constitutional amendment banning it
• Question types
• Open-ended (fill in the blank, short answer)
• Can get a lot of information, but
• Coding is time intensive and potentially ambiguous
• Close-ended (pick best answer, pick all that apply)
• Easier to code
• Response alternatives are the same for everyone
• Rating scales
• Used for how much judgments
• e.g., measures attitudes, agree/disagree
• Take care with your labels
• Range of scores, anchors
• Closed-ended
• What is the best thing about ISU? (choose one)
• 1. Location
• 2. Academics
• 3. Dorm food
• 4. People who sell things between Milner and the Bone
• What is the best thing about ISU?
• if closed-ended
• decide number/label of response alternatives
• should use odd number (5 or 7 best)
• labels should be clear
• decide scale
• rating PSY 231 is an important course in the major.
• Strongly Agree Neutral Disagree Strongly
• Agree Disagree
• semantic differential PSY 231
• Important _____ _____ _____ _____ _____ Unimportant
• Boring _____ _____ _____ _____ _____ Interesting
• nonverbal scale for children
• Point to the face that shows how you feel about the toy.
• Stage 7) Pre-testing the survey instrument
• Fix what doesnt seem to be working
• Stage 8) Selecting and training interviewers
• For telephone and in-person surveys
• Need to avoid interviewer bias
• Stage 9) Implementing the survey
• Stage 10) Coding and entering the data
• Stage 11) Analyzing the data and preparing a final report

PowerShow.com is a leading presentation sharing website. It has millions of presentations already uploaded and available with 1,000s more being uploaded by its users every day. Whatever your area of interest, here you’ll be able to find and view presentations you’ll love and possibly download. And, best of all, it is completely free and easy to use.

You might even have a presentation you’d like to share with others. If so, just upload it to PowerShow.com. We’ll convert it to an HTML5 slideshow that includes all the media types you’ve already added: audio, video, music, pictures, animations and transition effects. Then you can share it with your target audience as well as PowerShow.com’s millions of monthly visitors. And, again, it’s all free.

About the Developers

PowerShow.com is brought to you by  CrystalGraphics , the award-winning developer and market-leading publisher of rich-media enhancement products for presentations. Our product offerings include millions of PowerPoint templates, diagrams, animated 3D characters and more.

Non-Experimental designs: Surveys

Jan 01, 2020

160 likes | 169 Views

Non-Experimental designs: Surveys. Psych 231: Research Methods in Psychology. Sometimes you just can’t perform a fully controlled experiment Because of the issue of interest Limited resources (not enough subjects, observations are too costly, etc). Surveys Correlational studies

Share Presentation

• survey research
• response rate
• experimental designs
• experimental designs surveys
• psych 231 research methods

Presentation Transcript

Non-Experimental designs: Surveys Psych 231: Research Methods in Psychology

Sometimes you just can’t perform a fully controlled experiment • Because of the issue of interest • Limited resources (not enough subjects, observations are too costly, etc). • Surveys • Correlational studies • Quasi-Experiments • Developmental designs • Small-N designs • This does NOT imply that they are bad designs • Just remember the advantages and disadvantages of each Non-Experimental designs

What are they? • Questionnaires and interviews that ask people to provide information about themselves • Why conduct them? • Best way to collect some kinds of information: • Descriptive, behavioral, and preferential • e.g., demographic information, recreational behavior, and attitudes • To compliment experimental work • Good/common first step, can collect a lot of data about a lot of variables • Do not have to directly observe behaviors Surveys

Advantages • One can investigate internal events (for example, attitudes & opinions) • Can generalize about an entire population based on relatively small samples of individuals • Large amounts of data can be collected quickly with relatively little cost (effort, time, etc.) • But they’re often not as “cheap” as you may think Surveys

Disadvantages • Correlational: causal claims shouldn’t be made • Note: surveys are sometimes used as part of an experimental design, which may allow testing of causal claims • Non-response bias • Why doesn’t everybody respond? • Does response rate interact with variables of interest? • Large data sets are sometimes difficult to analyze • Self-reports may not be truthful • Response set - tendency to respond from a particular perspective • Social desirability bias (e.g., how a “moral” person would answer) Surveys

Stage 1) Identify the focus of the study and select your research method • What are the objectives of the research? • Is a survey method the best approach? • What kind of survey should be used? • Group administration • Mail surveys • Internet surveys • Telephone surveys • Face-to-face interviews • Focus group interviews Stages of survey research

Stage 2) Determining the research schedule and budget • Stage 3) Establishing an information base • Find out what’s been done, what’s known • E.g., Find other related surveys • Stage 4) Identify the sampling frame • The actual population that the sample is drawn from (as opposed to the ideal population) • Think of it as operationalizing the conceptual level population • Be aware of potential coverage error – when the sampling doesn’t lead to a good representativeness Stages of survey research cont.

Stage 5) Determining the sample method and sampling size • Review Probability and Non-Probability methods • Voluntary response method • Importance of sample size Stages of survey research cont.

Should leftover Halloween candy be given out to students who get an A on the exam? Call 123-4YES if you think YES Call 123-NONO if you think NO • A kind of convenience sampling methods commonly used • Problem: Typically only individuals with strong opinions respond, so the results are often extremely biased Daily show clip Another fun poll result Voluntary response methods

Sampling error - how is the sample different from the population? • Confidence intervals • An estimate of the mean or percentage of the population, based on the sample data • “John Doe has 55% of the vote, with a margin of error ± 3%” • Margin of error (that “± 3%” part) • The larger your sample size, the smaller your margin of error will be. • Which would you be more likely to believe • “We asked 10 people …” • “We asked 1000 people …” Importance of sample size

Sampling error - how is the sample different from the population? • Response rate • What proportion of the sample actually responded to the survey? • Hidden costs here - what can you do to increase response rates • Non-response error (bias) • Is there something special about the data that you’re missing (From the people who didn’t respond)? Importance of sample size

Stage 6) Designing the survey instrument • Question construction: How the questions are written is very important • Clearly identify the research objectives • Do your questions really target those research objectives (think Internal and External Validity)? • Take care wording of the questions • Keep it simple, don’t ask two things at once, avoid loaded or biased questions, etc. • How should questions be answered (question type)? Stages of survey research cont.

Problem: emotionally charged words Good Poor Was the FDC negligent by ignoring the warnings about Vioxx during testing and approving it for sale? Yes No Unsure If the FDC knew that Vioxx caused serious side effects during testing, what should it have done? Ban it from ever being sold Require more testing before approving it Unsure Good and poor questions

Good Poor Are you against same sex marriage and in favor of a constitutional amendment to ban it? Yes No Unsure What is your view on same sex marriage? I think marriage is a matter of personal choice I’m against it but don’t want a constitutional amendment I want a constitutional amendment banning it Problem: Biased in more than one direction Problem: Asks two questions Good and poor questions

Question types • Open-ended (fill in the blank, short answer) • Can get a lot of information, but • Coding is time intensive and potentially ambiguous • Close-ended (pick best answer, pick all that apply) • Easier to code • Response alternatives are the same for everyone • Rating scales • Used for “how much” judgments • e.g., measures attitudes, agree/disagree • Take care with your labels • Range of scores, anchors Survey Questions

Stage7) Pre-testing the survey instrument • Fix what doesn’t seem to be working • Stage8) Selecting and training interviewers • For telephone and in-person surveys • Need to avoid interviewer bias • Stage9) Implementing the survey • Stage10) Coding and entering the data • Stage11) Analyzing the data and preparing a final report Stages of survey research cont.

• More by User

Matched t test Experimental Designs

Matched t test Experimental Designs . Repeated measures Simultaneous Successive Before and after Counterbalanced Matched pairs Experimental Natural. Chapter 12. The one-way independent ANOVA An. O. Va. = Analysis of Variance. More than two groups .

640 views • 26 slides

Quasi & Non-Experimental Designs

Quasi & Non-Experimental Designs. Quasi-Experimental designs: Not quite true experiments because the different groups/conditions are not created by __________________. Groups or conditions are defined by _____________ variable or a ______ variable.

718 views • 14 slides

Quasi Experimental Designs

Quasi Experimental Designs. An Alternative to the True Experiment. Characteristics of Quasi-experimental designs. Goals and consequences Basic versus applied research Experimental Control Have manipulation of the independent variable May lack either or both of: Random assignment to groups

1.88k views • 12 slides

Experimental Designs

Experimental Designs. Campbell & Stanley’s (1963) three general types of experimental designs. Pre-experiments Quasi-experiments Full or true experiments. Key concepts.

432 views • 9 slides

Experimental Design & Analysis

Experimental Design & Analysis. Further Within Designs; Mixed Designs; Response Latencies April 3, 2007. Outline. Mixed, or split-plot, designs Response latency designs, a special case for split plots. Mixed Designs.

690 views • 11 slides

Experimental Design CSC426

Objectives . Overview of Experimental Design.Two-Groups Experimental Designs.Classifying Experimental Designs.Factorial Designs.Randomized Block Designs.Covariance Designs.Hybrid Experimental Designs. . 05/04/09. 2. Experimental Design. The most "rigorous" or the "gold standard" of all resear

628 views • 22 slides

Experimental Group Designs

Experimental Group Designs. Group Designs. Simple Group Designs one IV with 2 levels 2 levels can be independent groups (Exp + Cont) 2 levels can be repeated measurements (pre/post) Complex Group Designs one or more IVs factorial designs more than 2 levels on the IV more than 1 DV

632 views • 47 slides

Experimental Design

Experimental Design. The Gold Standard?. Today’s Goals. Identify issues of internal and external validity with various experimental designs Design an experiment for a given topic Critique advantages and disadvantages of different designs. To Review.

750 views • 37 slides

More Designs

More Designs. Section 4.2B. Block. Group of experimental units or subjects that are known before the experiment to be similar in some way that is expected to systematically affect the response to the treatments. Similar to stratified samples in surveys. Randomized Block Design.

259 views • 11 slides

Experimental designs

Experimental designs. Between- / Within-subjects . Between-subjects design Each partcipant takes part in 1 experimental condition Analysis of differences between groups of participants from different conditions Within-subjects design Subjects take part in all conditions

586 views • 17 slides

EVAL 6970: Experimental and Quasi-Experimental Designs

EVAL 6970: Experimental and Quasi-Experimental Designs. Dr. Chris L. S. Coryn Kristin A. Hobson Fall 2013. Agenda. Statistical power/design sensitivity Construct validity External validity. Statistical Power/Design Sensitivity. Types of Hypotheses. General forms: Superiority

527 views • 33 slides

Choosing Research Designs I

Choosing Research Designs I. Experimental and Quasi-Experimental. Experimental Designs. Experimental research design: The researcher has control over the experiment in terms of sample selection, treatment, environment, etc.

603 views • 15 slides

Chapter 8. Experimental Design II: Factorial Designs

Chapter 8. Experimental Design II: Factorial Designs. Chapter 8. Experimental Design II: Factorial Designs Chapter Objectives. Describe factorial designs using a standardized notation system (2x2, 3x5, etc.) and place data accurately into a factorial matrix to calculate row and column means

760 views • 27 slides

Chapter 9. Producing Data: Experiments. Experimentation. Recall the distinction between experimental designs and observational designs In experimental studies, the investigator exposes individuals to a treatment to ascertain its effects. Vocabulary.

257 views • 12 slides

Research Methods

Research Methods. Descriptive Methods Observation Survey Research Experimental Methods Independent Groups Designs Repeated Measures Designs Complex Designs Applied Research Single-Case Designs and Small-n Research Quasi-Experimental Designs and Program Evaluation. Experimental Methods.

611 views • 16 slides

Research Designs

Research Designs. Review of a few things Demonstrations vs. Comparisons Experimental & Non-Experimental Designs “IVs” and “DVs” Between Group vs. Within-Group Designs. Reviewing a few things… Kinds of bivariate research hypotheses (and evidence to support) Kinds of Validity

437 views • 25 slides

FACTORIAL DESIGNS (Treatment Design)

FACTORIAL DESIGNS (Treatment Design). Erlina Ambarwati. Parts of Experimental Design. Set of experimental units. Set of treatments. Rules by which treatments are assigned to experimental units. Measurements made on experimental units following application of treatment.

1.88k views • 43 slides

Web-Based Surveys Questions, Answers, and Designs

Web-Based Surveys Questions, Answers, and Designs . Kent L. Norman Department of Psychology Human/Computer Interaction Lab Laboratory for Automation Psychology http://lap.umd.edu. Reasons for Web-based Surveys:. Cost-efficiency, Dissemination-collection, Central control Automation

243 views • 16 slides

Overview of Study Designs

Overview of Study Designs. Study Designs. Experimental. Observational. Analytical. Group Randomized Trial. Descriptive. Randomized Controlled Trial. Cohort. Case-control. Cross-sectional. Ecological. Experimental v. Observational. “Gold Standard” Investigator controls exposure

743 views • 25 slides

Correlational Designs Causal Modeling Quasi-Experimental Designs

Correlational Designs Causal Modeling Quasi-Experimental Designs. Quasi means “seeming like.” Quasi-experiments superficially resemble experiments, but lack their required manipulation of antecedent conditions and/or random assignment to conditions.

1.18k views • 30 slides

Quasi-experimental Designs & Survey Research

Quasi-experimental Designs & Survey Research. Quasi-experiments. Hedrick, Bickman & Rog (1993) “a quasi-experimental design is not the method of choice, but rather a fallback strategy”

894 views • 21 slides

EVAL 6970: Experimental and Quasi-Experimental Designs. Dr. Chris L. S. Coryn Dr. Anne Cullen Spring 2012. Agenda. Regression discontinuity designs. Questions to Consider. Throughout today’s discussion, consider the following questions

358 views • 21 slides