Dear fellow seminaristianites,

I ask students to give peer feedback on short arguments submitted by their fellow students, in which they score them from 1 to 5 on four different aspects, and then explain their score and make suggestions. The rubric I developed for this describes what should be expected for each score. Typically what happens is that students mostly give each other fours and fives, with threes being reserved for the barely coherent.

In the fall, I plan to ask students to collaborate to develop their own rubric. I will ask them what sorts of things they think would be important to evaluate for a good bit of argumentative writing, and what would qualify something as below, above, or at average for each of those criteria. I would probably do this as a think/pair/share, before opening it up to a class-wide discussion in which I would write down and organize their suggestions.

The fact that students have a personal interest in what standard they will be judged by, as well as it being a low-stakes situation for sharing their ideas, should lead to a high level of participation, and I hope a greater commitment to the resulting assignments that the rubric will be applied to.

I am unlikely to have decided to try this without the example and feedback from Kyllikki Rytov in my group, who has successfully used this approach in her own classes.

The remaining issues I need to decide before implementing this are:

• How much guidance to give the students:
  • e.g.
    • Should I decree that the rubric scale will be 1-5, or let them decide?
    • Should I tell them the four characteristics/dimensions I have used before (rational persuasiveness, accuracy, writing mechanics, and originality), or let them come up with their own?
• How much time to allot to each step of this in class.
• Whether to break it up into more steps or fewer.
• How to make the instructions as clear and unintimidating and potentially fun as possible.

If anyone has advice or suggestions, I’d love to hear it!

The Artifact in Progress:

Instructions

For four weeks of the term, you will be asked to submit a short argument in which you take and defend a position regarding an issue covered in recent class readings and discussion. In the week after submitting an argument, you will also be asked to read and evaluate some of your fellow students’ arguments. Today’s task is to create the rubric for how you will score (or be scored) for these arguments.

[Give quick example of a basic rubric, e.g. “Handwriting” – 1=completely illegible, 2=can only be read with effort, 3=legible, 4=easy to read, 5=easy to read and beautiful to look at]

[Three minutes]

Take a few minutes now and think about what is needed, generally, in a good piece of argumentative writing.

Write these down.

If you have time, see if you can identify broad categories that could organize the items in your list. Draw lines, mark with colors, or put a symbol to mark things that you think belong to the same category.

[10 minutes]

Next, get together with a classmate or two and compare lists. Establish which of you will share them with the class.

See if you can identify and agree on four or five broad categories that cover the items in your lists (e.g. ‘good grammar,’ ‘correct spelling,’ ‘enjoyable to read,’ and so on might all fall under a category like “clear writing” or “good communication of ideas”).  These would be the ‘Dimensions’ in the left-hand column of the rubric. Write these down.

Discuss what you believe should count as a 1 (lowest), 2, 3, 4, and 5 (highest) for each of these dimensions on the rubric. Take notes on what you think the criteria should be for each score that could be given on the rubric. That is, if a “3” is given for a particular Dimension, what specifically does that mean, in terms of that Dimension? If you were given a 4, how should you understand it, or if you give a 2, how should the student receiving that score understand it?

[15 minutes]

Get together as a class and share your ideas. I will write down all of the suggestions, then consolidate and edit them to make the rubric you will be using for the Evaluations portion of this assignment.

Short Argument Evaluation Rubric

Dimensions	1	2	3	4	5

I have a series of low-stakes assignments in which students are asked in one week to submit a short argument on a recent course topic, and then in the next week they are asked to evaluate 3 other students’ short arguments, using the PeerMark assignment feature on Turnitin. All submissions and evaluations are anonymous (unless students deliberately put their names in the body of their work).

I have been tweaking this assignment model off and on for a few years, with mixed results, but this semester it is going very well. In their evaluations, I ask students to give the argument they are evaluating a score on various parameters and then to give constructive, helpful comments to explain their score and to help the arguer improve. I also ask them to present a counterargument.

I’ve been pleased with the effort students have been putting into these assignments this semester, but I am considering two changes in how I do this next time, and would love suggestions or comments on how best to carry them out (or for alternatives):

1. Have students create the rubric for what each score should represent (giving them a blank or pared down version to collaborate on completing).
2. Find ways to encourage follow-up or opening communication about the submissions after the initial argument/evaluation cycle. (I’m not sure how to do this without undermining or outright destroying the benefits of having everything be anonymous initially.)

Regarding 1, the evaluation questions and current rubric are below.

EVALUATION QUESTIONS (Those that require a scored response using the rubric are in bold.)

• As I understand it, the thesis is that: (fill in or cut and paste from the argument itself)

• How rationally persuasive is the author’s argument? Does it provide good evidence and reasoning to support the thesis?

Scale    Highest: very persuasive, Lowest: not persuasive

• What advice would you give for improving the rational persuasiveness of the argument? (Where, if anywhere, does the argument fail to rationally persuade? Or, if it is successful, what makes it work? Note that this is not the same as a counterargument; this is advice for how to improve the argument, versus a counterargument which gives reasons to reject the argument.)

• How accurately does the author present course material from readings or lecture? Are terms defined, philosophers’ views explained, and issues interpreted correctly?

Scale Highest: completely accurate, Lowest: mostly inaccurate

• Identify and explain any inaccuracies noted above. If you gave a 3 or less, clarify exactly what you think is wrong in the definitions or presentation of class material!

• Is the writing clear and well organized? Assess the quality of the writing itself (considering the grammar, spelling, style, and so on, rather than the content).

Scale Highest: very clear, Lowest: very unclear

• What suggestions would you make to improve the writing clarity or organization? Or, if you have no suggestions, identify what makes it successful.

• How original is the thinking shown? That is, does the author mostly repeat ideas from others, such as in the readings and class discussion, or does the author show independent thought, with new arguments or examples (or at least new twists or personalization of them)?

Scale  Highest: very original, Lowest: not at all original

• Provide the best counterargument(s) you can, which must include at least one piece of supporting evidence (NO questions, and NO mere disagreement).
• Optional: other comments you wish to share with the arguer? An overall ‘grade’ for the argument?

THE CURRENT RUBRIC: (The formatting seems to have gone a bit wonky as posted on the blog – it’s not this ugly, really!)

Argument Evaluation Rubric

	Needs Lots of Work/Poor   1	Needs Some Work   2	Average     3	Pretty good, with room for improvement   4	Excellent   5
Rational Persuasiveness	Not at all convincing; supporting evidence is absent, false, irrelevant, highly controversial, too general, or badly explained.	A little convincing; the argument needs more or better supporting evidence, the evidence needs better explanation, or the assumptions are controversial.	Adequate, with an average level of persuasiveness.	Fairly convincing, but assumptions could be made more explicit, or the evidence could be further developed in other ways.	Extremely convincing; the supporting evidence is specific, accurate, relevant, and well explained. Assumptions are either uncontroversial or supported.
Accuracy	There were many errors, misinterpretation, or false claims.	Some important points were wrongly reported or were misinterpreted.	Basically okay, but lacks detail or precision, or misleads on minor points.	Almost everything was accurate and appropriately interpreted.	Everything stated was accurate and appropriately interpreted.

Clarity and Organization	The meaning was very difficult to understand.	The meaning and/or connection between ideas were often unclear.	Relatively easy to follow, with decent grammar, etc., but should be edited to bring it above average.	The meaning was mostly clear, with some bits that could be re-written for greater clarity or logical flow.	The meaning is clear throughout, and the ideas are well-organized with good logical flow.  The writing is efficient and effective.
Originality	Arguments, examples, or ideas seem straight from the reading or class discussion.	Only minimal efforts have been made to personalize the arguments.	Average.	While perhaps a bit derivative, the author has clearly made the arguments his or her own.	Examples or ideas show innovative, independent thought.

I’m Amy Trautwein, from the Philosophy Department. I’ve taught a variety of courses at Baruch, such as Thought and Reality, Moral Problems of Life and Death, Critical Thinking and Ethics, and (the course I will be concentrating on for this seminar) Major Issues in Philosophy.

My courses usually have about 40 students in them, though I have taught jumbos with over 100 and seminars with only about a dozen.  

Major Issues usually has about 40 students enrolled in each section. Here is the course description and list of learning goals I usually include on my syllabus for the course (though I change the questions from time to time, depending on what I choose to focus on in a given semester):

“What does one do in philosophy? Philosophers look for questions, even where you may not have thought that questions could meaningfully be asked. We look for evidence for what are the most plausible answers even for questions that you may not have believed have answers at all. We all do philosophy sometimes, even if you may not have called it that.

This course will introduce a few of those philosophical questions.  We shall focus on questions regarding humans, our possible nature and our relation to the world.  Possible issues will include considering: what is human nature, if it even exists; if free will can exist and what it might be; whether reason can prove the existence of God; the nature of truth and knowledge; whether morality is just a matter of opinions, and if not, what else it might be; and what ‘minds’ are.

            In each case, the goals for you, as a student, are to:

1. gain a clear understanding of what is being asked regarding the issue;
2. grasp the positions some philosophers have developed to try to answer the question;
3. be able to identify and understand the premises and logical structure of the arguments given for those positions; and
4. evaluate these arguments and defend your evaluation with carefully reasoned arguments of your own.

A further goal is to develop your own answers to the questions.  However, answers that satisfy you might be years in the making.  For the purpose of this course, what counts is how well you grapple with the material presented.  The ultimate goal of this course is to improve your arsenal of thinking skills and broaden your approach to the world by careful examination of specific philosophical questions and answers.”

I have a peer review assignment for which students post a short argument on a recent course topic in one week, and the next week post evaluations of some of their fellow students’ arguments, which includes a mandatory counterargument against each of those arguments. My goals for these assignments is to have students think about the material more deeply, practice organizing and expressing their thoughts and defending their views, and also engage in metacognitive reflection about what makes for good arguments and clear communication. I would like to find non-grade-oriented ways to get students to feel more motivated to do the assignment for its own sake and not just as a rote exercise.