PLEASE SCROLL DOWN TO "ADD COMMENT". 

Below is the email message the Departmental Assessment Coordinator sent to the ENGLISH-L list on 1 March 2012:

Hi, folks. Paul and I have chatted about What to Assess Next. Since we don’t have general education assessment mandates this year, we have a chance to do something different.

Paul suggested that we focus in greater detail on one piece of our literary interpretation rubric, namely, this part:

- understand individual works, genres, and literary movements in relation to history and society

This outcome addresses a new requirement in the revised English major, and applying the outcome to this year’s assessment would allow us to create a “baseline” that we could measure against a few years from now.

So…unless you want to suggest  another aspect of our major that we need to assess more urgently, perhaps we could go with this. What we would do, then, would be to expand this outcome into its own rubric. I’ll be putting together a rubric for your viewing pleasure, but first I would like to solicit input from everyone in the department. What does it mean to “understand” works/genres/literary movements? What do “history” and “society” mean for us? Or should we use a term such as “culture”? And what statements of outcomes could we use that would apply to a typical set of papers from both an “early” course (170, 142, 213) or a later course (i.e., 310 or 358?)

I’d like to have your thoughts about this (think about it, for instance, when you’re on that SUNY-subsidized cruise to Belize over spring break) between now and the end of March. And, really, it would be great to get a wide range of suggestions and even a conversation about it, one that I would hope would reach a rough consensus by Tuesday, 27 March. You can share your comments and ideas at the following address on the wiki: https://wiki.geneseo.edu/x/9wGmB

By Tuesday, 2 April, I’ll post a rubric which you can comment on for the next few days, and then I should have the rubric ready to go by the following week.

 I will also need to ask for several volunteers who would be willing to apply this rubric to one of your later papers this semester — either a final assignment or a final exam. It doesn't take long to apply the rubric — we're talking about maybe 30-45 SECONDS per assignment. But the aggregate data can help us to observe trends.

We really didn’t have a good-sized sample last spring, just a few classes (including one of mine). So it would be good to have at least three volunteers from 100- and 200-level courses and three or four from 300-level courses. If you know in general that you have an assignment that will require students to put certain works into an historical/social/cultural context, I would invite you and cajole you and encourage you (and maybe visit you in your dreams) to join in.

As we should continually remind ourselves

— assessment is a requirement; we need to have these reports on file for Middle States accreditation

— assessment is the responsibility of everyone; the same volunteers over and over could give us a fairly odd axis of outcomes that might not represent what we are doing

--assessment can form part of studying and improving what we are doing in our major

I hope to hear from you (and I will send reminders) very soon!

Cheers

Graham

Your Very Own Assessment Coordinator

9 April 2012: AND NOW: The draft of the rubric, with some initial comments:

The draft version of the “Literature in Historical and Social Context Rubric” appears below. This is substantially Paul’s construction, along with some suggestions for revised language arising from department members’ discussion on the wiki.

Note that I have changed some aspects of Level 4 and Level 3 traits to try to capture what Paul was emphasizing about the centrality of historical/social context in a student’s essay. I was also trying to avoid the word “competent” on Level 4 because of its use on Level 3; thus I have suggested “makes a skillful and intellectually complex case for its thesis” in place of “makes a competent case for its thesis.”

I also want to acknowledge the comment Maria brought up about production of interpretation and the clarification of a student’s ideological position. This is an important comment, but as I’m seeing it right now, this is an issue that may play a significant role in the student’s self-reflection (and how we end up evaluating/assessing it).  So we should revisit that question. Right now, I think what we’re trying to do is to get an initial handle on students’ awareness of/use of history and social context, whatever the subject position and ideological stance a student’s writing may have.

At any rate, please have a look at this rubric over the next few days. If you have any suggestions, please let me know. I will then publish the rubric on the wiki on Monday the 16th.

Level

Traits

4 - Excellent

Essay proposes an interpretation of a text that puts the text in relation to some relevant aspect of history or society as a substantial element of an argument, or uses the text as evidence to illuminate some aspect of history or society, and makes a skillful and intellectually complex case for its thesis in a way that draws on evidence from both within and around (i.e., historical and social facts) the text.

3 - Satisfying

Essay demonstrates a competent attempt to interpret the text in relation to historical or social facts, or shows a somewhat nuanced understanding of how texts reflect or shape historical or social realities.

2 - Approaching

Essay refers accurately to historical or social facts in the course of interpreting a text, or competently makes a point about history or society using a text as evidence.

1 - Not Approaching

Essay makes no accurate reference, or no reference at all, to historical or social facts.


  


  • No labels

12 Comments

  1. Unknown User (okada)

    What does it mean to “understand” works/genres/literary movements? What do “history” and “society” mean for us? Or should we use a term such as “culture”? And what statements of outcomes could we use that would apply to a typical set of papers from both an “early” course (170, 142, 213) or a later course (i.e., 310 or 358?)
    For me, historical and social context are very important.  Since I approach many of my courses from the perspective of "these texts/authors/styles have been marginalized," the context is as important as the text itself.  So it's important to ask continually, why have these things been marginalized or ignored and what does that mean?  Understanding film movements, for example, such as American avant garde cinema means to understand the place of formal experimentation in relation to dominant media. Why would someone make a slow zoom across a room for 45 minutes?  This seems the antithesis of entertainment or classical Hollywood cinema.  Why did this filmmaker make this film?  Is it significant that it was done in the 1970s?  It means questioning assumptions of what cinema is and what it's for, the difference between profit making media and artisanal or personal cinema, not that these two examples of cinema are mutually exclusive.  Also, industrially, many of these filmmakers knew each other and helped each other distribute work.  Without artistic coteries and artists gathering around certain ideas and politics of particular periods, certain films would not be made.  Many, not all of these experimental films are in reaction to social and political currents and this is essentially to the understanding of why.

    Hope that helps Graham.

  2. I echo Jun's opening comments and would draw my examples from teaching courses focusing on American women writers ("scribbling women") and Native American authors.  As in the Underground Cinema example, the contexts are important to formal analysis, to interpreting meaning, and to understanding the ways that the texts are engaging and contesting other traditions.  Rather than adding other examples from my own courses, I've pulled a few course objectives from my syllabi that take a stab at articulating outcomes. To create the rubric, it might be helpful to look at the outcomes on syllabi for courses like 237.  But I don't think this is limited to courses examining marginalized writers, as I know historical context is just as important, say, to Rob when he is teaching Yeats and Irish literature. I've personally found these objectives difficult to articulate and would be interested in seeing how others have grappled with the language.

    From Engl 339 American Ways: Plotting Women

    Students will: 3) be able to situate literary works by American women writers in larger traditions, movements, historical, and cultural contexts; 4) be able to demonstrate an understanding of the theoretical and critical debates surrounding the interpretation of literature by American women writers.

    From Engl 237 Voices & Perspectives: Native American

    Students will: 1) think critically about the role of language and literature in Native American and American cultures, recognizing and understanding connections/distinctions between oral and written literary traditions; 2) demonstrate knowledge of the ways that Native American authors negotiate complex moral, social, aesthetic and political issues in creating a written literary tradition within/against Western generic conventions and American cultural myths.

  3. "understand individual works, genres, and literary movements in relation to history and society." As broad a brush as this statement is, it describes more accurately than anything I could say about how I organize literature courses, so I am happy to see us focus on this outcome. (Disclaimer: I am teaching no English courses this semester -- HUMN and HONR).   I tend to be relatively proscriptive (even authoritative, at times) about the terminology and fundamental information about literary periods and movements about date ranges and basic "definitions" of literary periods and movements, but fortunately most of these terms (and even date ranges) are sufficiently flexible, imprecise, subject to scholarly debate, so I can sometimes engage students in discovering/playing with the loose ends and frayed seams of "Romanticism/Romantic Era," or "The Renaissance," etc.

    I tend to use the term “culture” as an all encompassing  contextual term that, eventually, leads to more particular considerations of “political history” (rulers and governments), “social history” (gender, race, class, economics, ethnography,), and arts & intellectual history (state of theater, publishing, art or aesthetics, literary or theatrical movements, philosophy).  For me, the term “culture” supports these (somewhat) more specific concepts.

    While the context of texts is important to any course or specific discussion of texts, ultimately I emphasize textual analysis (against relevant context(s)) in having students engage in analysis, interpretation, and ‘original’ responses and reflections.

  4. I'd like to see us keep this simple. We might look for some very basic indicators of whether students, in their essays, discuss texts in isolation from historical and social contexts or go some ways towards contextualizing the works. The scheme for evaluating the essays on this outcome could be as simple as this:

    Level

    Traits

    4 - Excellent

    Essay proposes an interpretation of a text that depends on the text's relation to some aspect of history or society, or uses the text as evidence to illuminate some aspect of history or society, and competently makes the case for its thesis in a way that draws on evidence from both within and around (i.e., historical and social facts) the text.

    3 - Satisfying expectations

    Essay makes competent use of historical or social facts in pursuing an interpretation of a text, or shows a somewhat nuanced understanding of how texts reflect or shape historical or social realities.

    2 - Approaching expectations

    Essay refers accurately to historical or social facts in the course of interpreting a text, or competently makes a point about history or society using a text as evidence.

    1 - Just not getting it

    Essay makes no accurate reference, or no reference at all, to historical or social facts.

    It should take no longer than about 15 seconds to decide, for each paper you grade, which of the four numbers above applies to it. Fifteen seconds times 35 papers comes to 8.75 minutes. Are you capturing all the rich complexity of each student's strengths and weaknesses as a writer? No. When the results from multiple classes are combined, do we as a department have some kind of handle on whether our students, in the aggregate — no matter whose course they're in — "get" what we mean when we tell them that texts don't exist in a vaccuum? I think we do.

    I can't stress enough that the point of this isn't to prove anything to anyone. This is why I couldn't care less whether someone in Erwin or Albany has a response to the information we gather. I want to know whether our students "get it." If they don't, or only get it minimally, I want us to do something about it. If it took a lot of time and effort to get this information, I might resent having to do it, even though I'm the beneficiary. But it doesn't.

    1. Paul's suggestion reminds of the HUMN essay rubric's "Historical context" category.  I've been looking at it lately, as it is really the only rubric I've used that considers what Graham outlines: 

      Historical context

      4 Argument is informed by a sophisticated, in- depth knowledge of history beyond required Humanities texts

      3 Argument constructively utilizes historical knowledge, as presented in required Humanities texts

      2 Argument demonstrates some knowledge of history, whether cited or indirect

      1 Argument shows little or no evidence of historical knowledge, even indirectly

      I always thought it interesting how the best examples moved "beyond" the texts, the works read and analyzed, although how the argument arrived there wasn't quite acknowledged.  The phrase "evidence from both within and around" seems a better way to suggest the student's analytic relationship to historical overview and the texts themselves.  

  5. I assume all of us, no matter what our critical and theoretical interests and assumptions are, want students to be aware of the relationship between literary texts and some of the most relevant historical and cultural contexts. I say "relevant" because the larger area of "context" is so broad as to approach infinity . . . One can't really, for instance, understand and appreciate the poetic revolution proposed or defined in Wordsworth's famous Preface to Lyrical Ballads unless one understands his new poetics in their relationship to the French Revolution. As some of his contemporaries already noticed (especially Hazlitt),  there is a clear (if not necessarily cause and effect) connection between the two (political and poetic) revolutions. Because students often lack even the rudiments of the necessary historical and cultural background, they need to be made aware of it in some fashion or other. For instance, when I have taught D.H. Lawrence's (much anthologized) story, "Odour of Chrysanthemus" in English 170, I've found that even some of the best students are initially clueless about the importance of the early-twentieth century setting of the story and the cultural context of gender and social class (in this case, English working class) that define much of what happens in the story.

    Having cited two of my favorite authors, let me also note that historical-cultural context should include the author's biography or life: in Wordsworth's case, the young radical poet's animus against the aristocratic regime in England was partly grounded in his childhood--the fact that his father, who worked as a political agent for the leading aristocrat-politician in he North of England, had not received his salary for several years, and when he suddenly died and left his five children orphans, Lord Lowther refused to pay the debt he owed to the Wordsworth family/children. The adult poet's fervent support of the revolution in France is clearly grounded in part in the traumatic experience of his childhood. In the case of Lawrence, the connection between his life and his work is so immediate as to be impossible to ignore (the first part of Sons and Lovers is, as he acknowledged, almost entirely autobiographical). So  biography is also or actually an important part or sub-unit of "historical context." 

    At the same time, historical context is in many ways  a slippery and tricky and uncertain category, as well as one that can easily be oversimplified and turned into a kind of pastiche. As Goethe points out in his satire on the scholarly life in Faust, the spirit of the past that historians think they see is mostly a reflection (or construction?) of their own minds. . . . you know, the flattened out "the Elizabethan world view" sort of thing. Historical and  cultural context is especially difficult and challenging an issue with respect to complex works, like Shakespeare's plays. For instance, ghosts appear in some of them (as in Hamlet and Macbeth), and are seen (or not seen, as the case may be) by several characters, but how relevant or important to our understanding of the play(s) is it to know what Shakespeare's culture believed about ghosts? When we are dealing with non-Western cultures, this issue becomes even more difficult--for instance, Maxine Hong Kingston's The Woman Warrior: Memoirs of a Girldhood Among Ghosts." When I was in China, I actually consulted with some Chinese graduate students about the Chinese belief in "ghosts" and found what they told me somewhat helpful but not interpretively decisive. Thus I tend to be leery or suspicious of generic pronouncements about cultural norms and practices in other times and places . . .  like the role of women in  the early modern period, when these statements are made, for instance, without reference to crucial factors such as social class, country, etc. 

    Having rambled on, let me close by saying that I like Paul's 1-4 rubric or formula above, though it could even be simplified:

    3. Essay shows a sophisticated and nuanced awareness of the importance of the significance of the relevant historical and cultural context(s) to the work(s) in question;

    2. Essay shows some awareness etc.

    1. Essay shows little or no awareness etc.

    1. The idea of relevant historical & cultural contexts seems really important.  The HUMN rubric for "Interdisciplinarity" uses the phrase "to the subject under discussion" to suggest a kind of focus that seems analogous, but I don't think that's really a model.  So, for Paul's rubric, could three read: 

      Essay makes competent use of <relevant> historical or social facts in pursuing an interpretation of a text, or shows a somewhat nuanced understanding of how texts reflect or shape historical or social realities.

      And four could say, "Essay proposes an interpretation of a text that depends on the text's relation to some <relevant> aspect of history or society. . . ."  Maybe one and two could stand as is.  

      While simplicity is a virtue, using four categories allows the rubric to follow some of the other college-wide models, and conveniently suggests an equivalency to our usual grading levels: excellent, very good, average, below average.  

  6. I appreciate the broader discussions of the issue here and I believe I'd be redundant were I to attempt my own contributions to those comments.  On the other hand I am supposing that Paul's straightforward approach to the issue comes to grip with the matter immediately at hand in fashioning an appropriate rubric.  There my inclination is to endorse the simplicity of Gene's approach; but somehow every rubric I've seen lately tends to favor 4- or 5-part responses.  Whichever way we go, may I tinker with the foregoing suggestions just a bit?  In my opinion both Paul's and Gene's might be improved at the "higher" end of their scale.  Paul's "Excellent" level overstates the centrality of the contextual dimension of a given assignment, I believe.  To give the rubric enough flexibility, we might wish to recognize the apt use of historical/social/cultural contexts without insistence that a given analysis utterly "depends" upon such factors.  Gene's versions match the spirit of this recommendation, but my own (too low?) standards shy away from "sophisticated and nuanced"; I'd be quite impressed by "competent" or "informed"  awareness of significant historical and cultural contexts.

    I had briefly toyed with a suggestion involving the use of non-"literary" resources as part of a research project--an action rather easily quantifiable and measurable as it will appear (or not) in the "Works Cited" section of a research paper.  This is a concept I've sometimes use as a basis for particular research assignments.  But I recognize that it is no true measure of the intellectual activity we're attempting to measure. 

    1. Would "acknowledge" work for "depends?"

      Essay proposes an interpretation of a text that <acknowledges> the text's relation to some <relevant> aspect of history or society, or uses the text as evidence to illuminate some aspect of history or society, and competently makes the case for its thesis in a way that draws on evidence from both within and around (i.e., historical and social facts) the text.

      Hmm.  I don't know.  

      1. How about, "Essay proposes an interpretation of a text that puts the text in relation to some relevant aspect of history or society," etc.?

        I would argue for an "excellent" (or perhaps "exemplary") category that insists that the essay actually, and centrally, makes a connection between a text (or texts) and history/society, rather than simply bringing history or society in at some point in the discussion.

        An essay that brings history/society in competently would still be "satisfying expectations." And for assessment purposes, "satisfactory" is truly that. (That is, not code for "middling.")

        In general, I prefer a rubric that defines each level qualitatively; i.e., that identifies a distinctive trait or traits for each level. I find this more useful than a rubric that essentially identifies only one trait and then sets the levels quantitatively, by asking "how much" of that trait you find (a lot, a little less, even less than that, none).

        When I read student papers or evaluate other student work, it seems to me that the differences between levels of achievement are not a matter of simply more or less of one thing that I'm looking for. Rather, the differences reflect different kinds of qualities or different constellations of qualities. Rubrics necessarily (and usefully, I think) simplify these qualities somewhat, but in my view should still try to capture them.

        What I like about the discussion we're having is that I think we're making progress on describing the traits/qualities we care about. Thanks, everyone! More!

  7. Graham,

    You and I met earlier to talk about the rubric, but only now am I joining the discussion.  I like Paul's rubric, but I'm wondering whether only the site of PRODUCTION of the text should be emphasized...  Isn't it equally important for students to attempt to characterize this historical moment and, as far as possible, their ideological standpoint?  Reception Studies shows that there's no such thing as "the text itself"-- there are as many Great Expectations as reviewers and critics have constructed the novel since publication -- not to mention film adaptations...    Just a thought.

    As usual, I volunteer my three classes this semester.

    Happy Easter/Passover/Sunday!

  8. Level

    Traits

    4 - Excellent

    Essay proposes an interpretation of a text that puts a text (or texts) in relation to some relevant aspect of history or society as a substantial element of an argument, or uses the text as evidence to illuminate some aspect of history or society, making a skillful and intellectually complex case for its thesis in a way that draws on evidence from both within and around (i.e., historical and social facts) the text.

    3 - Satisfying

    Essay demonstrates a competent attempt to interpret a text in relation to historical or social facts, or shows a somewhat nuanced understanding of how texts reflect or shape historical or social realities.

    2 - Approaching

    Essay refers accurately to historical or social facts in the course of interpreting a text, or competently makes a point about history or society using a text as evidence.

    1 - Not Approaching

    Essay makes no accurate reference, or no reference at all, to historical or social facts.