Testing Effects and the Solomon Four Group Design
When conducting a study with a pre-test/post-test design (i.e., a repeated-measures study), one of the major threats to validity that you will face is the threat of testing effects. A testing effect occurs when scores on the post-test are influenced by simple exposure to the pre-test. For example, say you are trying to determine if a prep course is effective at increasing students’ SAT scores. You conduct a study in which you compare students’ SAT scores from before they enrolled in the prep course (pre-test) to their scores after they complete the prep course (post-test).
Any differences you find between the pre-test and post-test scores may be (at least partially) attributable to a testing effect. When the students take the SAT for the first time, they gain experience that may affect their performance on the next test, regardless of whether they take the prep course or not. For instance, the students get to see the types of questions that appear on the test and get to experience the pressure and stress of the test environment. These and other potential factors may cause the students to perform differently the next time they take the test.
Need Help with your Analysis ?
Schedule a time to speak with an expert using the link below.
So how do you account for testing effects in such a study? One simple method to control for testing effects is to use a Solomon four group design. In a Solomon four group design, the participants in the study are randomly assigned to four different conditions: a) intervention with pre-test and post-test, b) pre-test and post-test with no intervention, c) intervention with post-test, d) post-test with no intervention. Conditions a and b are what you would see in a typical pre-test/post-test design with a control group. Conditions c and d replicate conditions a and b except no pre-test is included. Having these additional conditions allows the researcher to determine if any changes occur simply due to the pre-test. This is done by comparing condition b (pre-test and post-test with no intervention) to condition d (post-test with no intervention), and by comparing condition a (intervention with pre-test and post-test) to condition c (intervention with post-test).