During AY 2019-2020, our department was in Phase 1 of our new assessment protocol, which is aligned with the ACTFL World-Readiness Standards for Learning Languages.
The assessment methodology in place from 2014-2018 were adapted from the 2012 ACTFL Proficiency Guidelines, but the rubrics used often produced inflated results and they were cumbersome because a numeric score was entered for each individual student.
In an effort to streamline the process, the Assessment Committee ask instructors to submit a report that provides a global assessment of the entire class, including whether or not there is any correlation between level of students' success gaining specific knowledge assessed and their relative level of success in the course as a whole. In order to have a clear picture of the student learning outcome assessed, instructors upload examples of student work that is "meeting standards", "approaching standards" and "developing." Finally, instructors briefly reflect on how they taught the specific skill assessed and what students did or did not respond to in the teaching of that skill.
To facilitate this process the Assessment Committee created the following:
- shortened rubrics that focus on whether students are "meeting standards", "approaching standards" or "developing"
- sample tasks that can be modified for use in the assessment
- Google Forms survey where all material will be submitted
Intended Student Learning Outcome #1 |
|---|
Proficiency in Cultural Knowledge, based on the language learner’s level as suggested by the ACTFL guidelines on Intercultural Communication. ACTFL has created a series of “Can-Do” statements to describe which communication tasks language learners at a specific level can consistently accomplish. Assessing cultural knowledge by means of rubrics using ACTFL’s “Can-Do” statements aligns our assessment protocol with the ACTFL World-Readiness Standards for Learning Languages. The “Can-Do” statements are as follows: Novice-High students (the equivalent of 100-level students): “In my own and other cultures, I can identify cultural products and practices to help understand perspectives.” Intermediate-Low/Mid (200-level students): In my own and other cultures, I can make comparisons between products and practices to help understand cultural perspectives Intermediate-High (300-level students): In my own and other cultures, I can explain some diversity among products and practices and how it relates to perspectives. |
| Alignment with institutional mission and goals | Please type text in boxes below. They will expand as needed. |
|---|---|
| College Mission Reference | Students who attain Intermediate-Low/Mid proficiency in cultural knowledge in another language and culture can make comparisons between their native culture and the one they are studying. Students who attain Intermediate-High proficiency in cultural knowledge in another language and culture can identify, analyze, explain and interpret cultural attributes, thereby allowing them to forge connections to people from other countries. Such students understand and value cultural diversity. Proficiency in cultural knowledge at the Intermediate level, as outlined by the ACTFL guidelines therefore aligns with the college’s mission to inspire students to be "globally aware citizens" who value inclusivity by respecting the uniqueness of cultures and individuals. |
| Relevant College Values Reference | Our assessment results have revealed that students achieve the greatest level of cultural knowledge proficiency by engaging in active learning with authentic cultural texts and media. The teaching and learning that happens in the language classroom--active and student-centered--that leads to proficiency in cultural knowledge aligns with the college mission the "rigorous curriculum, transformational learning experiences" that are expressed in the College Values of Learning and Creativity. Additionally, students who achieve a high level of proficiency in cultural knowledge are often those who study abroad. Our department faculty members actively encourage students to participate in study abroad programs as a way to deepen their linguistic and cultural knowledge; proficiency in cultural knowledge therefore promotes “ethical global citizenship.” |
| Relevant Strategic Plan Focus Area | This intended student learning outcome was developed in alignment with the college’s GLOBE framework. A student who has attained Intermediate-High proficiency in cultural knowledge has done so as a result of integrative and applied learning. Proficiency in cultural knowledge enables intercultural communication in a language other than English, which is a tangible and valuable practical skill. Therefore the relevant Strategic Plan Focus Areas include Learning, Access and Success, and Advancing the Public Good. |
| Relevant Program Goal or Learning Outcome | At least a third of our Program Learning Outcomes include a Cultural Knowledge component. These include: recognition of social and regional linguistic variation; knowledge of distinctive features of the history, institutions, economy, society, culture, etc. of the country or countries where the target language is spoken; critical understanding of the patterns of social interactions, as well as the meanings, attitudes, values, and ideas of the target cultures; knowledge of the main authors, literary trends, and styles connected with a given course’s focus and time period. |
| First Means of Assessment | Please type text in boxes below. They will expand as needed. |
|---|---|
Procedures & Criterion for Success | Written assessment of cultural knowledge in the form of tests and quizzes, such assessments often include a reading comprehension component and a written component. Students are asked to read a short text and respond in writing to questions related to cultural knowledge. A successful student at the Intermediate-High level will be able to: Analyze and explain in writing distinct cultural products and practices from written texts; Identify, compare and interpret in writing distinct cultural and/or historical attributes of written texts. |
| Assessment Results | We collected data from 169 students from a variety of courses in Spanish, French and German. Although we moved online during the spring semester, responses didn’t diverge much nor did they have an impact on the collection of the data. In general, students met standards as described on the rubric we created. |
| Reflection on Results | This was the first academic year for our new assessment protocol. One of our assessment innovations was to replace a cumbersome numbers-heavy evaluation system with a clear, concise rubric inspired by the ACTFL “Can-Do” statements. One immediate result is that providing instructors with a clear, common and not cumbersome rubric was key to help instructors and encourage completion of the evaluation process. The variety of resources that instructors used to incorporate cultural knowledge into their courses showed that teaching cultural knowledge focuses on a broad variety of tasks, including speaking, writing, lexical variations, general cultural understanding, the pragmatics of language and comparing cultures. Therefore, instructors’ inventory ranged from selections found in textbooks to other formats such as movies, visual art, magazines, newspapers or news reports. The combination of the format with the function (in writing, speaking, watching or listening) creates a rich approach to target this learning outcome. Overall, all courses and students were exposed to a wide range of materials on cultural knowledge in the target language. The results show that TV shows, news broadcasts, videos embedded in the textbook, round table discussions and music videos increased student interest in the learning of culture, that’s to say: real and current events about the countries and the languages they are learning about. One of the instructors noted that “interest increased when the discussion is contextualized at their interconnectedness level (global interdependence).” This aligns with our learning outcomes and Geneseo’s Global Awareness and Engagement (GLOBE learning outcome). What doesn’t work to enhance cultural knowledge is pure reading and note taking without context. |
| Second Means of Assessment | Please type text in boxes below. They will expand as needed. |
|---|---|
Procedures & Criterion for Success | Oral communication related to cultural knowledge is tested daily by different means: on-the-spot questioning of individual students, group work, short dialogues and presentations. A successful student at the Intermediate-High level will be able to: Orally explain distinct cultural products and practices from written texts and/or media; Orally identify and compare distinct cultural and/or historical attributes of written texts and/or media. |
| Results | We collected data from 169 students from a variety of courses in Spanish, French and German. Although we moved online during the spring semester, responses didn’t diverge much nor did they have an impact on the collection of the data. We looked closely at the correlation between the relative level of students' success gaining cultural knowledge and their relative level of success in the course as a whole, there is a slight differentiation between fall 2019 and spring 2020. 75% of the students evaluated during fall 2019 showed a strong correlation while 50% of them did it for spring 2020. Some factors that might explain the lower correlation during the spring semester: “Their assessment was in writing and also taken in an online setting due to the pandemic which resulted in changes in content delivery. I choose [sic] "weak" correlation above because the oral communication skills' practice was not the same after switching to online.” |
| Reflection on Results | Switching to an all-online mode of delivery during Spring 2020 crystallized the huge value added of in-person learning and time on task in the communicative language classroom; the more students hear and practice speaking the target language, the more confident and comfortable they are in taking linguistic risks, which leads to the reward of higher proficiency in cultural knowledge and intercultural communication. As we move to a 4-credit course model we are buoyed by the positive impact the additional classroom time will have on our students’ proficiency in cultural knowledge and intercultural communication. Throughout our assessment of cultural knowledge we have discovered the following about how we teach cultural knowledge and how our students best learn this skill:
|
Intended Student Learning Outcome #2 |
|---|
| Alignment with institutional mission and goals | Please type text in boxes below. They will expand as needed. |
|---|---|
| College Mission Reference | |
| Relevant College Values Reference | |
| Relevant Strategic Plan Focus Area | |
| Relevant Program Goal or Learning Outcome |
| First Means of Assessment | Please type text in boxes below. They will expand as needed. |
|---|---|
Procedures & Criterion for Success | |
| Results | |
| Reflection on Results |
| Second Means of Assessment | Please type text in boxes below. They will expand as needed. |
|---|---|
Procedures & Criterion for Success | |
| Results | |
| Reflection on Results |
Intended Student Learning Outcome #3 |
|---|
| Alignment with institutional mission and goals | Please type text in boxes below. They will expand as needed. |
|---|---|
| College Mission Reference | |
| Relevant College Values Reference | |
| Relevant Strategic Plan Focus Area | |
| Relevant Program Goal or Learning Outcome |
| First Means of Assessment | Please type text in boxes below. They will expand as needed. |
|---|---|
Procedures & Criterion for Success | |
| Results | |
| Reflection on Results |
| Second Means of Assessment | Please type text in boxes below. They will expand as needed. |
|---|---|
Procedures & Criterion for Success | |
| Results | |
| Reflection on Results |
Summarize departmental discussion of assessment results. (Discussion Guidelines) Have you made, or are you considering, any changes (to courses, programs, co-curriculum, use of other campus resources) in light of assessment results? If so, please describe the changes below.
Based on our previous years’ assessments, our department has decided to propose moving to a 4-credit course model. General Education students will be provided with more “time on task” per course in the target language. It will also increase cultural knowledge by enabling the integration of media and technology that will increase the quality of students’ time on task-- essential steps on the path to the GLOBE learning outcomes of “skill in another spoken language” and “global awareness and engagement.” For our department’s minors and majors, 4-credit courses will open the door to more integrative and applied learning opportunities within the classroom through workshops, digital media and project-based learning. Also, we are increasing collaboration with other departments, programs and offices across campus in cultural activities such as the Hispanic Heritage Month, the French week, and Octoberfest.
Parts II and III of the CAC Assessment Report:
II: Identifying of the Learning Outcome Goals of Your Academic Unit for next year.
Name 1-3 learning outcome goals you will assess next year. This next year's goals may be based on assessment from last year or the planned rotation of student learning outcomes assessment. By archiving this information on the wiki using this form, you can access these goals next year.
Goal 1: Proficiency in Cultural Knowledge
How you will measure your success:
Success will be attained if the majority of students “meet” or “approach” standards on our culture tasks rubric. We also measure success through instructor reflection on their teaching; this reflection combined with the assessment results enables us to “close the assessment loop.”
Goal 2: Oral Proficiency
How you will measure your success:
Success will be attained if the majority of students “meet” or “approach” standards on our oral proficiency rubric. We also measure success through instructor reflection on their teaching; this reflection combined with the assessment results enables us to “close the assessment loop.”
Goal 3:
How you will measure your success:
III: Linking Academic Unit Learning Outcome Goals to the College's Mission, Values, and Strategic Plan
Please tie your goals for the upcoming year to Geneseo's declared values (Learning, Creativity, Inclusivity, Civic Responsibility, and Sustainability) or to the more granular Focus Areas and Objectives outlined in the Geneseo 2021 Strategic Plan.
Goal 1:
This goal is aligned with the following objectives from the four focus areas of Geneseo’s values: Pursue curricular innovation; Improve support to faculty and staff to explore innovative approaches teaching, learning, and research; Provide a welcoming and inclusive environment; Strengthen our engagement with local and regional communities including alumni; Connect scholarly resources of the College to appropriate projects in the surrounding community and other partner communities;
Goal 2: This goal is aligned with the following objectives from the four focus areas of Geneseo’s values: Pursue curricular innovation; Improve support to faculty and staff to explore innovative approaches teaching, learning, and research; Provide a welcoming and inclusive environment; Strengthen our engagement with local and regional communities including alumni; Connect scholarly resources of the College to appropriate projects in the surrounding community and other partner communities;
The attached document below represents the assessment data that was presented and discussed at the Languages and Literatures department meeting on September 23, 2020. During that meeting, the Curriculum and Assessment Committee proposed to assess proficiency in cultural knowledge and proficiency in oral communication during the AY 2020-2021. Some faculty expressed their reticence to assess oral proficiency since the pandemic has changed modes of course delivery and methods of teaching. Some faculty requested more discussion of whether or not it is advisable to assess oral communication this year; we will have this discussion at the next department meeting.