SOCL-211 Statistics for Social Research
Department of Sociology Assessment 2014-15
Prepared by Michael Restivo, Ph. D.
One of the intended educational outcomes in Sociology is that students will demonstrate knowledge of basic descriptive statistics, multivariate analysis, inferential statistics, and use of statistical software. To assess this set of learning outcomes, three (3) sections of SOCL-211 – Statistics for Social Research were examined from Fall 2014 (two sections) and Spring 2015 (one section). Direct measures were taken from students’ work including homework assignments, exams, and data analysis projects using statistical software. Students’ work were evaluated as either satisfactory or unsatisfactory across five knowledge domains: 1) descriptive statistics, 2) probability, 3) inferential statistics, 4) multivariate analysis, and 5) statistical software. A total of 108 students were assessed, with 62% (n=67) meeting all five criteria and 83% (n=90) meeting four out of the five criteria.
SOCL-211 Statistics for Social Research is designed to introduce undergraduate students to knowledge and practice in the application of quantitative methodology to social scientific research. Students learn how sociologists engage in the process of research by using theory to make assumptions, create statistics and interpret the results from these inferences. The course also focuses on statistical literacy and the ability to evaluate and critique the numbers that students encounter in academic work and beyond.
The measurement of sociological phenomena, the collection of data, the choice of procedures, and the interpretation of results are fundamentally intertwined. Therefore, the aims of the course include: 1) how to determine the appropriateness of various procedures given the research problem, 2) how to correctly apply this procedure once one is chosen, 3) how to clearly and accurately interpret results, and 4) how to identify limitations, potential misuses, and misinterpretations of these techniques.
The course also has a lab portion designed to introduce undergraduate students to the mechanics of computer-aided statistical analysis of quantitative data. The computer program used is the Statistical Package for the Social Sciences (SPSS), which is available to students using the CIT Virtual Computer Lab. Students will develop basic proficiency in SPSS that can be applied to many types of research. Students can optionally install on their own computers a free equivalent of SPSS called PSPP, which is an open-source alternative to SPSS that maintains compatibility with SPSS data and syntax files. The most important feature of PSPP is that it is free to download and use regardless of what type of computer it is being used on. This ensures that the software proficiency that students gain in this course will be available to them throughout their time at Geneseo and beyond.
Knowledge domains assessed
Students’ proficiency of statistical knowledge was assessed across the following five domains: 1) descriptive statistics, 2) probability, 3) inferential statistics, 4) multivariate analysis, and 5) statistical software
Direct measures were assessed from each student’s work on homework assignments, lab exercises using statistical software, data analysis projects, and examinations. Students proficiency, based on an assessment of their work on these measures, was evaluated as either satisfactory (meeting the criterion) or unsatisfactory (not meeting the criterion) in each of the five knowledge domains listed above.
A total of three sections of SOCL-211 from Fall 2014 (two sections) and Spring 2015 (one section) were assessed. Overall, 109 students enrolled and 108 students completed the course. Of these students, 14 were Sociology majors at the time of the course (13%) and 94 were majors in other disciplines (87%); these included Communication (13%), Political Science (8%), International Relations (7.5%), as well as students who had yet to declare a major (12%).
Results and discussion
Of the 108 students assessed, 67 students (62%) received a satisfactory evaluation on all five criteria and 90 students (83%) satisfied four of the five criteria. The knowledge domain with the most students who did not meet the criteria successfully was “multivariate analysis,” where more than a quarter of all students failed to meet the criteria. This suggests that greater efforts should be made by faculty teaching the course to convey this material more clearly and effectively.
A comparative analysis of students who took the course in Fall 2014 compared to students who took the course in Spring 2015 revealed no statistically significant difference in the number of criteria met (χ2 = 6.34, p = 0.274). Similarly, a comparison of Sociology majors compared to all other majors showed no statistically significant difference in the number of criteria met (χ2 = 8.10, p = 0.151). Note that because there are only a small number of Sociology majors (n=14), the expected frequencies for Sociology students who did or did not meet the requirement in any of the five knowledge domains will be low. Therefore a chi-square test may not be appropriate to use in this instance. As a robustness check, a Fisher’s exact test used; the test yields a direct calculation of the probability that the number of criteria met is not related to the students’ major. The result of the Fisher’s exact test is p = 0.235. Thus in both instances, we conclude that there is no statistically significant difference in outcomes between Sociology and non-Sociology majors, nor any differences in learning outcomes between semesters.
Overall, the results of this assessment indicate that a majority of students complete SOCL 211 Statistics for Social Research with the intended level of statistical knowledge. As noted above, there is room for improvement in teaching “multivariate analysis,” which remains the most difficult and least well-grasped topic in the course. Students who go on to conduct research might benefit from taking an intermediate level statistics course focusing on multivariate analysis, if one was offered. Other than in this area, based on this assessment no major changes to the Sociology department’s statistics courses are warranted at this time.
Table 1. Summary of knowledge assessment