Department of Communication
Closing the Loop on Effective Speaking Skills
2002-2012
Introduction
One of the requests made by SUNY Geneseo’s Assessment Committee is for academic departments to “close the loop” on learning outcomes, using information from ongoing assessment data to reach a better understanding of what has been learned about a particular learning outcome, and what steps might be taken in the future.
Historical Overview – Effective Speaking Skills, 2002-2012
- An educational outcome to assess speaking skills was formulated in academic year 2002, and defined as follows: “Students will demonstrate oral competency in the ethical and effective presentation of original ideas.” Communication majors in the graduating class of 2002-2003 who had been observed by at least three faculty members when presenting ideas in coursework or in related co-curricular occasions were assessed on organization of material, delivery, and overall affect (attitude and motivation of the speaker). A four-point scale, (1= unsatisfactory…4=outstanding), was used by the faculty evaluators. All students in the survey achieved at least a satisfactory/acceptable rating across the three criteria, showing the highest mean score for affect (2.96). While the department was relatively satisfied with results, there was interest in refining the assessment procedure to better define the criterion of organization and provide more longitudinal data.
- A population of graduating Communication majors in 2004 was evaluated for oral competency using the method developed in the previous year. Oral competency of seniors was assessed using the same criteria and a four-point scale with one alteration to the evaluation rubric; the criterion of organization was more specifically defined as “the ability to use evidence to support main points, and organization of ideas and materials in all parts of the speech.” Once again, all students surveyed achieved at least a 2.0 rating across the three criteria --- 5% over the acceptable standard for success. Although not statistically significant, the department noticed a slight decline of the mean score for the sample across the three evaluative criteria. The decision of the department was to continue monitoring oral competency in the next academic year, improve the scale descriptors for the three criteria to more closely match those of the college’s General Education rubric, and encourage the faculty to put more emphasis on organization of speeches in their public address assignments, since the mean score (2.48) for this criterion was the lowest of the three measured among graduating seniors.
- Data was not collected for the outcome of effective speaking skills in academic year 2005-2006.
- In academic year 2006-2007 the outcome of effective speaking skills (“effective oral presentation”) across a population of 127 student majors in three classes was assessed. The department compared speaking skills of students in introductory and advanced classes to determine if there were specific areas for improvement. Of particular interest was the ability of students in upper-division courses to demonstrate stronger speaking skills than their peers in the introductory public address class. Assessment revealed that student skills in introductory and advanced classes were similar. Some differences, however, were noticed; for instance, in the advanced classes students were minimally better at using audio-visual and supporting material. Review of the data led to two conclusions: 1) some students, whether in introductory or advanced classes are struggling with appropriate use of audio-visual and supporting materials, and 2) proper incorporation of citations in speeches produced the lowest assessment scores, no matter the course level. It was agreed that instructors, particularly those teaching the introductory public address course should give more instructional attention to citation of sources and appropriate use of supporting materials. Refinement of the assessment process was discussed to allow for comparison of faculty and student perceptions of effective speaking skills, and improved selection of upper-division classes to measure the various facets of this learning outcome.
- In 2007-2008, the outcome of effective speaking skills was not evaluated. The Department assessed the learning outcome of basic writing skills.
- During the 2008-2009 academic year the Department of Communication engaged in a self-study, at which time the faculty discussed six assessment outcomes. Two conclusions were reached for the outcome of effective speaking skills: 1) seek to improve measurement so that a comparison can be made between student and faculty perceptions of students’ speaking effectiveness, and 2) continue to measure three oral discourse skills in the introductory speaking course to determine whether source citation, incorporation of supporting material, and effective use of audio-visual aids have improved over time, particularly as more instructional attention has been given to these skills.
- In year 2010-2011, “effective speaking skills” were assessed in two 300-level Communication courses (Comn 362 and Comn 391). Applying the college’s oral discourse rubric to group presentations in these courses, the department found that students met expectations for organization, expression and presentation. The instructor’s evaluation of these criteria was satisfactory but lower than student evaluations. Organizational ability, while assessed as satisfactory, was deemed to be the weakest skill. Specific observations about each of the criteria are found in the ongoing assessment report for that year. The assessment approach used was useful, but the department concluded extrapolating the results of oral discourse competency to students in all upper-division courses was not warranted.
- Ongoing assessment of “effective speaking skills” continued in 2011-2012 with focus placed, once more, on the required introductory course (Comn 102). Student evaluators and the instructor used a local assessment instrument that specifically earmarked organization, research, source citation, and use of audio-visual aids. Expectations were met for these criteria, but the instructor’s evaluation was somewhat lower for organization. While it appears the added instructional emphasis given to organization, research, citation, and use of audio-visual aids is producing better results in the introductory course, uncertainty remains about the ability of students to retain and apply speaking skills as they move through their academic program, from 100 to 300-level courses.
What has Been Learned?
Ongoing assessment of the learning outcome, effective speaking skills (oral discourse), has produced these findings.
- Although the description of the learning outcome changed somewhat over the years, as did methods of evaluation, longitudinal data reveal…
- Most student majors consistently meet expectations for effective oral discourse, more specifically the college’s criteria of “organization,” “expression,” and “presentation.”
- Students tend to be weakest in organizational skill, hence instructional emphasis has been given to this aspect of public address in the required introductory course (Comn 102).
- There has been modest improvement of skills of organization, research, citation, and use of audio-visual aids among students completing the introductory course. The same conclusion cannot be made for students in 200 and 300-level courses.
- It remains unclear whether students retain and/or apply effective speaking skills practiced in the introductory course as they progress through their academic program.
- Future ongoing assessment of this learning outcome should pay particular attention to the student retention and application of effective speaking skills across courses in the Communication curriculum. In other words, after the experience of a required introductory speaking course, do our majors continue to improve as presenters and evaluators of public address? How might we measure that? What should be measured beyond the sweeping criteria of “organization,” “expression,” and “presentation.”