Rubrics, Peer Response, and Student Self-Assessment

These are very rough notes; not liveblogged, but posted as I walked through the lobby and got a free wifi connection.

SRO

Scott Geisel, “Analytical Steps toward Creating Self-Sufficient Writers: Rubrics, Peer Response, and Self Assessment”

(Comp and business writing.)

There isn’t much research into peer review. Cindy Selfe: “I can’t believe we’re still asking students to do that.” Observes that teachers often comment that the peer review doesn’t always go well. What makes peer review difficult?

Students are more committed when accountable. A required comp course requires much more direction and motivation than a course with committed writers.

Students complain that their peers don’t catch their mistakes.

+ – ? (plus, minus question) works well for some students, but we are asking them do things that are difficult even for us.

Knowing their drafts will be public meant some students spent more time on the draft.

Students are largely capable of making at least quantitative assessments; “Here is a rubric; look for these things, are they there or are they not there?” Qualitative comments emerge.

Assumption: that these processes should transfer to self-assessment. Writing center tutors: “I can’t believe how much my writing has improved by helping others.”

Students get confident after the quantitate evaluation, so that’s a good time to guide them towards more qualitative.

Rather than ask them “What would you do differently” after it’s all over, get them to think about that while they still have time to make changes.

Students turn things in for a grade “with a preface.”

Get students to prioritize changes; get students to think about these things while they are writing, rather than when they are done.

Reality check — comments need to be more substantial; students don’t comment on big holes in peer drafts.

More realistic expectations: More opportunities to think about their drafts and revise leads to better drafts. (Woah — he blew through those last two slides in 5 seconds. Would have liked to see those conclusions for longer.)

Debrief with an activity where the student names the peers who gave them specific feedback. Students then do a self-assessment based on the peer feedback.
Reflection — explain what you changed and why; how it improves your paper in terms of the assignment objectives; student writes a self-analysis, and peers respond to that self-analysis.

Critical Preface — students pull out details and quote themselves, demonstrating what the reader will find that meets a specific assignment goal.

Sheri Rysdam and Lisa Johnson-Shull, “The Ink Left Behind: Failure, Nonsense and Cruelty in Peer Responses to Student Papers”

Johnson-Shull perceives a culture of negativity in responding to student writing. Begins by flashing a Wordle for 2 seconds — I was typing and missed it.

Looked at “labels of failure and mean comments in an anonymous peer-to-peer feedback situation”. Despite being taught a “civilized” peer response rubric, students tended towards cruelty in their feedback. 42% of teacher comments were “corrections”

Can students learn anything measurable from learning the Assignment, Focus, Organization, Support, Proofreading (AFOSP) rubric. Students first asked to respond cold to peer papers, then after having learned the rubric. Students who learn the rubric can still be mean.

Rysdam: introduced a sample student essay that “has some problems.” 39% of peer responses were counted as “Failure/Meanness.”

Quoted some of the 600 “Failure/Meanness” comment (as the audience moans in frustration at what we are seeing and hearing) Students were LOOKING for failure — they perceived it as their job to identify problems, and students would sometimes write “I couldn’t find anything wrong wit hit.”

Johnson-Shull wonders, if they are so negative did we teach them? We did set the example that it’s OK to make corrections without explanation, and profs may have their own keys for the squiggles and circles that the researchers marked as “non-sequitur,” but where does the negativity come from? 1% of teacher comments focus on failure, but almost 40% of peer.

Students knew they were part of a research project that was designed to test their ability to use a rubric; they seemed to think the researchers wanted them to demonstrate the negativity.

Ed White reflects on the irony that teachers of lit and writing (who think of themselves as stewards of great thought and revolutionary ides) are so universally seen a “picky pedants” who are mostly interested in correcting.

Where does the vehemence come from when students act as anonymous responders to their peers?

Audience Q — there is a value in this even if they are being cruel; it’s not so much how much stuff they put on their peer’s paper, but how they reflect on what they get from their peers.

Audience Comment: noted the anonymous nature; compared to anonymous comments online.

Geisel notes that he works hard to make sure his peer reviews are NOT anonymous.

“Authentic Responses” were very low, both in teacher samples and peer samples; that seems to be the missing “kindness” that would get us heard when we make substantial suggestions.

The review process teaches more to the reviewer than the reviewee.

End class by showing some reviews anonymously, asking whether the comments are helpful.

We (faculty) are used to being interrupted, when we read, by things we need to comment on; students who are not used to reading work to a rubric may respond more strongly than we do when they notice a flaw in a peer paper.

Johnson-Shull mentioned a study in which one group of kids were praised for being tidy recyclers, and another was not; both groups were given juice boxes; the tidy-identified kids threw their straw wrappers away, while the control group just dropped them. Suggested we as educators could benefit from setting students up so they are conscious of what they do well.

Geisel says it takes longer to write positive comments; he tells his students he will give praise orally but will focus on what the student needs to improve. Johnson-Shull asks whether there is a psychological benefit to praise that we don’t consider. Constructive feedback is not necessarily mean, so kindness does not mean ignoring problems.

Audience: Move from “It’s bad, here’s how to fix it” to “It’s good, here’s how you help your reader understand you.”