Intended Learning Outcome:
“Students will demonstrate an understanding of the types of social research.” More specifically, they will:
- distinguish between exploratory, descriptive, explanatory & evaluation research.
- distinguish between observational or experimental research
- be able to identify dependent and independent variables and assess the measurement reliability and validity of these variables
- make critical evaluations of existing research
In the spring Semester of 2011 twenty-four students enrolled in Socl212 Sociological Research took an Assessment Pre-test at the beginning of the semester. The results of that test were compared to the final exams of the twenty-four students. (The course had an enrollment of twenty-five, but one student did not take the pre-test.)
The mean proportion correct on the pretest was 0.58 (SD = 0.09). Half the students got at least 0.58 of the pretest points available.
The mean proportion correct on the final exam was 0.71 (SD = 0.16). Half of the students got at least 0.75 of the final exam points available.
The correlation between the proportion correct on the final exam and the proportion correct on the pretest is positive but not very strong. (r = 0.326). The pretest proportion correct only 'explains' about 10% of the variation in final proportion correct. Virtually all of the explained variance is due to the presence of an extreme value for one student (0.35, 0.36) who did very poorly on both tests. If this student is removed from the analysis the correlation virtually disappears. This student put very little effort into the course and thus the instructor gave him an E grade.
The red x data points are the students who had a higher percentage correct on the pretest than on the final, while the blue o data points are the students who scored at least 0.3 higher on the final than on the pretest.
The four students who improved the most between the two tests were a sociology major, a dual mathematics/sociology major, a dual political science/sociology major, and a sociology major with a pre-med concentration.
The four students who did not improve at all between the two tests were two sociology majors, a dual psychology/sociology major, and a communication major. In general these four were weaker than average students. The instructor gave the communication major a D grade, which would not be passing for a sociology major.
The mean change in proportion correct between the pretest and the final exam was 0.13 (SD = 0.15). Half the students did at least 0.15 better on the final exam than on the pretest.
On the final exam students tended to do better on the questions based on chapters in the text book, except for the chapter on sampling. They did not do well on the questions about positivism and interpretative sociology, which had been taught by lecture without required reading. Collectively, they got 65% credit for the two research evaluation questions, while they got 75% credit for the questions on text chapters and lecture material.
The students improved most on factual information about sociological research, but they did not perform as well when asked to evaluate descriptions of social research (one quantitative and one qualitative) on the final exam. Their evaluations of research did not make full use of the tools presented in the course. If the sociology department wants students to be able to read and make critical evaluations of existing research, then students must be given more practice making evaluations.