|
Experimental Design Tutorial: (Continued)
To access the quick reference guide and the first part of the experimental design tutorial simply click on the following links. Experimental Design Quick Reference Guide
Experimental Design Tutorial: Part 2Let's start with the logic of the experimental method: If two groups of participants are equal in all respects save one and are not similar in respect of a behaviour that is being measured, then the difference between them must be attributable to the one way in which they were different (J.S Mill). Hopefully by exploring this logic in relation to the word search experiment you'll see that it’s actually relatively straightforward. When we ran the experiment we had two groups of participants (some in PC lab 1) and some in (PC lab 2). Now for the time being, let’s accept that these two groups of participants were equal in all respects save one. This of course begs the question, what was the one way in which the two groups of participants were not equal? The answer is level of teacher presence. When the students were doing the puzzle in PC lab 1 (my lab), the teaching team made their presence felt as much as possible, walking up and down each bank of computers, looking over the students shoulder, essentially being as conspicuous as possible. In PC lab 2 (colleagues lab) the teaching team did the opposite i.e. they remained as inconspicuous as possible i.e. showed no interest in what the students were doing at all. Now if it transpired that students’ in PC lab 1 and PC lab 2 were not similar in respect of the behavior that was being measured. In our case the time taken to complete a word search puzzle, then according to the logic of the experimental method, this difference must be attributable to level of teacher presence. In other words Level of teacher presence (the cause) made a difference in the time taken to complete the puzzle (the effect). So the first thing to note about the experimental method is that it is fundamentally concerned with establishing cause and effect. Indeed it’s the only research methodology that allows you talk about cause and effect in relation to what is being investigated. In addition to understanding the logic of the experimental method you’ll also need to understand the language that accompanies it. So let's take each concept listed on the quick reference guide in turn and where possible, link it to the word search puzzle experiment.
As the definition states it’s the prediction of the outcome of the experiment. Now the hypothesis is the starting point in the process of experimentation, so when we were thinking about a simple experiment, we needed an idea that we could test and this is what we came up with. High levels of teacher presence in PC lab 1 will cause students to perform a word search puzzle more quickly than students in PC lab 2. Now because we’ve predicted the way in which behavior will change, i.e. it will get faster, we’ve actually formulated a one-tailed experimental hypothesis. If we’d simply stated that high levels of teacher presence will cause students in PC lab 1 to perform a word search puzzle differently than students in PC lab 2. We would have formulated a two-tailed hypothesis because all we’re saving here is that behavior of interest will change. The decision to choose a one-tailed or a two-tailed hypothesis depends on how confident you feel in predicting the way that behavior will change. We were confident enough to go for a one-tailed experimental hypothesis because a large body of research evidence suggests that the presence of others will improve performance on simple tasks. Another type of hypothesis you need to be aware of is the null hypothesis sometimes denoted as the H0, now this simply states that any observed differences between groups were down to chance. The idea behind it, is that depending on the result of statistical testing at the end of the experiment you will either reject or retain the null hypothesis. Put simply the bigger the difference between groups, the less likely that the difference is down to chance and the more likely you are to reject the null hypothesis. Continuing with the quick reference guide let’s have a look at the independent variable (the IV). This is the one factor that is different between conditions. Now the IV is the thing that you as the experimenter manipulate in order to see if it causes a change in behavior and as already mentioned it was level of teacher presence. So what about the dependent variable (the DV)? The aspect of behavior measured? Again as previously mentioned it was the time taken to complete the word search puzzle. Hopefully you’re beginning to see that although the experimental method uses unfamiliar terms like the H1, H0, IV and DV the things they allude to are not actually that complicated. Let’s go back to the logic of the experimental method and highlight the independent and dependent variables. If two groups of participants are equal in all respects save one (The independent variable) and are not similar in respect of a behavior that is being measured (The dependent variable) then the difference between them must be attributable to the one way in which they were different (The independent variable).
Well clearly this is impossible to maintain when dealing with human subjects for the simple fact that we’re not clones, we’re all different and we bring our individual differences with us when we take part in an experiment. For instance, some of the students will have had more sleep than others and this may have affected their performance i.e. they didn’t finish the word search puzzle as quickly as they might have done if they'd had a proper night’s sleep. Now this shouldn’t be a problem providing we are only talking about one or two people, and this kind of thing is referred to as a random error because things like number of hours sleep vary randomly in a population of people and it’s something that cannot be eliminated. It’s when random errors become constant errors that you’ll find yourself saying Houston we have a problem. Remember the cornerstone of the scientific method is to establish cause and effect and we do this by manipulating the independent variable to see what effect it has on the dependent variable. Now imagine if it transpired that all the students in PC lab 2 had been to the same party the night before and only had on average 2 hours sleep? Whereas; all the students in PC lab 1 had fallen out with the person throwing the party so they didn’t go and so they ended up having a normal night’s sleep. You now have no way of knowing whether students in PC lab 1 performed the word search puzzle more quickly because of level of teacher presence (the IV); or whether they performed the puzzle more quickly because they’d had more sleep and as such were more alert. In this example sleep has become a confounding variable. It’s confounded your results and the experiment is ruined. This somewhat frivolous example was used to introduce you to another integral aspect of the experimental method i.e. control. In the experimental method we accept that random error exists. We also accept that random error produces extraneous variables. We must, therefore, always do our best to ensure that extraneous variables do not become confounding variables. The simplest way to do this is to randomly allocate people to different conditions. So in our sleep example for instance, random allocation would ensure that all the party going tired people don't end up in the same group. This leads us nicely into the 3 main types of experimental design because as you’ll see each has strengths and weaknesses in relation to control issues. Experimental design simply refers to the way in which participants are deployed during the experiment. Before we look at the 3 designs I just want to quickly outline the difference between the experimental group and the control group. The experimental group is where you expect the predicted behavior change to occur (in our case PC Lab 1 i.e. high level of teacher presence) and the control group acts as the baseline so that this change can be assessed (PC Lab 2 i.e. low level of teacher presence).
Depending which book you read this design may also be referred to as independent groups, independent samples or between groups. The independent subjects design is the one we employed during the word search experiment. In this design participants are divided into entirely separate groups on the basis of random allocation, meaning that each participant has an equal chance of being allocated to either the experimental group or the control group. Let’s say we have 60 participants, we could put the numbers 1 to 60 in a hat and get participants to pick a number. Participants who pick an even numbers would go into the experimental group and participants who pick an odd numbers would go into the control group. Each participant will provide us with a single score. It’s then a case of looking at all the scores in the experimental group and comparing them with all the scores in the control group.
This is also referred to as the related samples design or the within groups design. In this design the participant performs in both the experimental condition and the control condition. Each participant, therefore, provides us with two scores or paired data. It’s then a case of comparing all the paired data scores to see if there’s a difference between the conditions. Now the main strength of this design over the independent subjects design is that you don’t have to worry as much about individual differences confounding the results of your experiment because if you think about it, each participant acts as their own control. In other words they take their random variability i.e. number of hours sleep, personality etc. with them across the conditions so that they cancel each other out. The danger of this type of design is that order effects will confound your results. Because each participant is being asked to do something twice it’s possible that they will perform better in the second condition because of practice or that they perform worse in the second condition because of fatigue or boredom. Therefore, we must ensure that the order of the two conditions is counterbalanced. Counterbalancing is a technique employed to ensure that half the subjects perform in the experimental condition first and the control group second. And half the subjects perform in the control group first and the experimental group second. Counterbalancing does not get rid of order effects but it does makes sure that any possible confounding effects cancel each other out. Independent subjects and repeated measures are the two most common types of design in experimentation and the thing to note about them is that their strength lies in the weakness of the other. With independent subjects design you don’t have to worry about order effects because you aren’t getting people to perform twice; but in the repeated measures design you are this is can be a real problem. However, in the independent subjects design you have to worry about individual differences between conditions; but in the repeated measures design you don’t because people take their differences with them, thereby nullifying any possible effect.
In this design participants are arranged into pairs on the basis of a pre-test. For instance you might pair people up who have a similar IQ. Members of the pairs are then allocated randomly to either the experimental condition or the control condition. It’s then a case of comparing the paired data scores to see if there’s a difference between the conditions. In Conclusion This tutorial has covered a lot of material and it would be worth going through at least a couple more times. If some of the concepts still seem a little abstract just keep trying to relate them to word search experiment. Please bear in mind that the tutorial only covers simple experiments where you only have 2 conditions i.e. the experimental group and the control group and where you only have one independent variable and one dependent variable. Experiments can be more complex than this you can have more than two conditions and you can have multiple independent and dependent variables but the logic remains the same. Finally I'd like to conclude with a definition of an experiment because having completed the tutorial it should make more sense. An experiment is a study of cause and effect. It differs from simple observation in that it involves deliberate manipulation of one variable (the independent variable), while controlling other variables (extraneous variables) so that they do not affect the outcome, in order to discover the effect on another variable (the dependent variable) (S. Heyes 1986).
Essential ReadingResearch Methods and Statistics in Psychology by Hugh Coolican
Book Description The latest edition of this market-leading textbook has been updated and revised to embrace current developments in this area of psychology. It remains a comprehensive survey of research methods in psychology today, with clear and detailed explanations of statistical concepts and data analysis. It also covers the full range of experimental and non-experimental methods and explores the ongoing quantitative-qualitative debate among researchers. It reexamines issues surrounding validity; including confounds, quasi, field, and non-experiments, and includes a new section on ethnographic methods. See following link for more details: Research Methods and Statistics in Psychology Back To Top Of The Page
|
