Incorporating Peer Assessment in Adult Education

by Tara Fenwick and Jim Parsons
Department of Educational Policy Studies
University of Alberta, Edmonton, Alberta, CANADA T6G 2G5
tara.fenwick@ualberta.ca

This is an early version of the article "Peer Assessment Without Tears: Making it Work", appearing in the Journal of the Alberta Association of Adult and Continuing Education, May 1999. (also available in The Art of Evaluation: A Handbook for Educators and Trainers by Tara Fenwick and Jim Parsons, published by Thompson Educational Publishers, Toronto, 2000.)

I conceive the major contribution of evaluation to be a heightened awareness of the qualities of life so that people can become more intelligent within it. -- Elliot Eisner

Peer assessment of adult learning enjoys sporadic spins of popularity in the workplace, in post-secondary education, and in community-based programs (Black, 1993; Davie, 1997). Whether one is assembling ‘formative’, developmental feedback or summative ratings to an individual or group, peer assessment invariably is considered to be one of the many means available to collect, interpret, and communicate evaluative information.

We all know the reasons why peer assessment is such a great idea in theory. Peer assessment helps create more democratic spaces for work and education, because it interrupts the top-down power flow set in place when a facilitator or boss is sole evaluator. Peer assessment provides those necessary alternate perspectives which balance the evaluation’s bias. Peers often point out practical issues from their own experience that may be blind spots for another evaluator, and offer a smorgasbord of rich feedback for learners. Peer assessment continues to be an important dimension of educator development (Heimlich and Norland, 1994), leadership development programs (Congor, 1997) and learning organization initiatives (Senge, 1999). Besides the benefits to the person being evaluated, the activity of assessing our peers can help us as the peer evaluators develop a key learning ability: the skill to evaluate oneself and others regularly, critically and appreciatively, using defensible criteria, and to form follow-up action plans for further learning.

The Problem With Peer Assessment

But in our society, peer assessment is not a natural part of our everyday communication. We are usually anxious to sustain warmth and harmony in a relationship, and steer clear of saying or doing things which may create lasting tension and awkward relations. Many people struggle when they are forced to give critical feedback to others, and many suffer when they are told things they interpret as negative. Goleman (1998) claims that our emotional awareness of self and others, and our ability to articulate precisely what we feel and perceive, are critical dimensions of our relationships at work and at home that need much more serious attention. We don’t always know what to look for, or how to interpret what we see in order to create a reasonable judgment. We tend to avoid the whole situation if we possibly can.

Problems can also develop when the learning activity of peer assessment is combined with summative judgment. For example, instructors may incorporate a grade derived from peer assessment with a grade they have awarded to a student. Sometimes there is discrepancy between the peer rating and the instructor’s rating for various reasons: peers anxious to support one another may award ‘inflated’ grades, or award unfairly low grades (Marienau, 1994). But in adult contexts, learners’ perspectives are viewed as offering valuable input. An instructor must think carefully before simply adjusting a peer’s assessment to be more in line with the instructor’s judgments.

So, our skills as assessors of our peers are not naturally strong. Most of us need to consciously think about what to do and how, and to practice peer assessment before we can be effective enough to be truly helpful to others. Facilitators, supervisors, and community development leaders cannot simply expect people to be effective evaluators of their peers because they are told to do it. They often need help.

Four Ways to Improve Peer Assessment

This article is intended to provide some help. We show four main processes in peer evaluation, along with some ideas for helping people improve their ability to participate meaningfully in these four processes. The four are subsumed in two main activities of peers assessment: observing and communicating.

Observing

• learning how to recognize one’s own role and limitations as an observer
• learning how to observe, interpret and assess what one sees, using criteria

Communicating

• learning to express one’s judgment in helpful ways to a peer
• learning how to receive, interpret, and act on feedback provided by others

These four processes usually unfold in situations of peer assessment such as these:

Individual -- One person provides face-to-face or written feedback for a peer, either anonymously or with full disclosure of evaluator

Group -- Together provide feedback for one individual through discussion.

Individual -- Evaluates a group he or she has worked with, through written feedback (Refer to the sample provided below: Evaluation Guide for Cooperative Groups).

Group -- Evaluates themselves as a group through discussion.

Group -- Evaluates another group, through discussion or written feedback.

Here are the four processes, each described through a series of suggested activities or discussion ideas to try with a group of learners.

1. Learning to recognize one’s role and limitations as a peer observer

• Discuss with people how the presence of an observer can shape the reality that is observed. The observer is never just a fly on the wall. By simply being there the observer changes the everyday actions of the person being observed into a performance for an audience. Wheatley (1994) reminds us, "Our perceptions of people and events shape the reality we then end up struggling with so much . . . Every time we go to measure something, we interfere."
  
  
• Discuss how an observer, no matter how attentive, can only focus on a tiny amount of the multiple processes going on as the observed person interacts with his or her environment. First, the observer is influenced largely by what he or she considers novel or important in a situation. Second the observer is human, with a very limited sensory system that cannot process the innumerable data available. Therefore much happens that the observer misses, or misinterprets.
  
  
• Invite people to consider themselves and others a ‘critical friend’ to their peers. Each person has the capacity to offer one piece of knowledge to their peers: an authentic response based on their own unique but limited and highly subjective perspective. This response is valuable simply because it broadens and enriches the information available.
  
  
• Try leading people through an activity that compares their own perspectives with their peers, and helps sharpen their sense of criteria. Following are three suggested activities:

Taste-Test: In our evaluation workshops, we sometimes ask people in small groups to taste-test different kinds of donuts or cookies. Each group must develop criteria, rate the items using this criteria, then present and defend their choice for ‘top pick’. This exercise invariably alerts people to the role played by personal preference and individual perception in any assessment.

Compare Ratings: Show a group of learners two or three short video clips of people performing activities demonstrating the kinds of skills the learners are trying to developing themselves. (For example, student teachers watch clips of teachers presenting information or working with students; managers watch chairs at meetings or staged samples of manager-employee interactions; employment counselors watch clips of staged interviews, etc.)

Ask viewers to jot down evaluative comments after the first clip, then share their comments. Most find that while there are some points of agreement, each viewer focuses on different aspects which influences their judgment. Our discussions also find that each viewer offers a different perspective which can alter the way one interprets what is seen.

After the first video, give learners a simple set of criteria to use, or have learners together develop a list of four or five benchmarks they consider important. Then watch all the clips, applying the criteria. The discussion following the viewing is usually rich with insight about the difficulties of observing, the dilemmas of comparing external criteria against the complexities of a particular incident, and the need for important information about context and the performer’s perspective which is lacking in this exercise, but crucial to understand what is being seen.

Read-Around Groups: To bring to the surface one’s inner sense of ‘what is good’, a Read-Around-Groups activity is helpful. People in groups quickly skim through a pile of writings by their peers, and rate each item. Then they examine their top picks to determine what traits made these the best, and what criteria they were using to decide. From these criteria they develop a rating scale that they apply to new cases. This can be adapted to any type of peer performance, by having people recall or observe samples of different work skills.

2. Learning how to observe, interpret and assess what one sees, using criteria

• Attentive, focused observation is not easy for many people. Everyone is susceptible to making assumptions that may blind them to significant details. Some people build perceptions more through intuition than by using their senses, possibly interpreting what they see so deeply from within their own rich inner worlds that they may not be fully present to what is happening. Learning to compare a list of criteria against what one is observing is difficult, partly because it’s hard to consciously attend to many different facets of the performance at once. Give people opportunities to practice observing and interpreting their peers’ ideas and actions. Newman (1991) offers many valuable suggestions for sharing perceptions of two people in ways which invite both into critical reflection of the experience they have both shared.
  
  
• Perceiving details, and describing specific details, takes practice. Keep emphasizing the need for clear, specific description of a peer’s performance. People often leap to interpretation and evaluation before noting carefully the evidence.

Note-Taking: A useful exercise when we are learning how to observe a peer’s performance is take longhand notes. First establish a focus for the observation, then simply record everything we see and hear. Then we should verify these notes with the person being observed, attending particularly to those ‘critical incidents’ upon which our assessment might be based. We should ask, Did you and I notice the same critical incidents? What did I miss in my observations that you as performer felt was important? Then we should check our interpretations with the person we observe. We ask, What did it mean for you when...? Why did you ....? And we share: This is what I thought was going on when you ....Here’s what I felt when you.....

Inter-Rater Reliability: Try having a group of people go through the process of establishing inter-rater reliability, using a particular rating scale. First have them become familiar with the items on the scale, and talk about their expectations for each item. Give them examples and benchmarks for the items. Then have them view and assess a few examples of performance, using the scale. Stop and compare their ratings. Raters present reasons for their ratings, and discuss discrepancies.. Then they view a few more samples, trying to make their ratings consistent with one another.

Ladder of Interpretation: Use the Ladder of Inference activity with a group, but make time to discuss the results and talk about the lessons in this activity for peer assessment. Chris Argyris (1993) created the Ladder of Inference exercise to help people uncover their interpretive assumptions. Jot down a brief description of a ‘critical’ incident with another person in which you were an observer or participant. Then focus on a small piece of that incident: Draw two columns lengthwise down a page. In the left column write down, in sequence, exactly what was said and done in the incident. In the right-hand column opposite each item, write how you interpreted the speech or action. Then -- and this is the hard part -- talk it through with the person you observed. Share your interpretations, and find out what the other person was thinking during the same incident. This activity immediately shows how much meaning we infer from very little observed evidence, ladling on our own expectations, assumptions, and personal understandings of what things mean.

3. Learning to express one’s judgment honestly and sensitively to a peer

• Palmer (1998) writes eloquently about the ‘grace’ of a connective, intimate community in which authentic learning and teaching can unfold. Remember that until a comfortable, mutual trust is established in a community, you cannot expect people to disclose their most honest opinions. They must feel assured first that their ideas will be received with respect, and without retaliation. This takes time. Trust unfolds through many conversations, in which people have opportunity to test out their ideas, assess the consequences, and gradually risk disclosing more and more.
  
  
• Peer assessment is effective when it unfolds in a caring relationship. that preserves the other’s dignity while sharing one’s experiences honestly. In such a relation the overall attitude of the assessor to the one being assessed is: I appreciate and value the unique contribution you bring. Here is my experience of what I observed you do/say. People may need reminding of this.
  
  
• Learners need clear criteria, and clear indicators or benchmarks of the criteria, to be able to provide helpful summative assessment for a peer.
  
  
• Discuss openly with learners the many power dynamics circulating in any group of people which can make expression of truly open, honest feedback difficult and sometimes impossible. It is naive to pretend that everyone in a group enjoys equal voice, status, and respect for listeners. people in any group are affected by each other. We are attracted to some and we resist others as we build various relationships -- these naturally affect the ways we observe and respond to one another. We want to establish identity and recognition in any group, and sometimes we struggle more to do so -- these dynamics also affect our observations and responses to one another.
  
  
• Make clear the difference between "personal response" and "criteria ratings". In criteria ratings, peers compare what they observe each other doing with a set of external criteria. In personal response, peers offer "I" statements, expressing what they personally felt and understood in response to what they saw. Both kinds of feedback are valuable. State clearly which kind of feedback peers are to offer in a particular situation.
  
  
• Learners may need reminding to avoid giving prescriptive advice to a peer. Stick to describing what was observed, offering personal responses to this observation (I felt. . . , I thought . . . , I assumed . . . .), or showing how the observation compared to the external criteria. The point is to share observations and personal responses, then show how these observations compare with the agreed criteria. Schon (1987) offers a model of feedback provision which builds a scaffold for learners through descriptive observation, enabling them to reflect more precisely on their their own performance.
  
  
• Allow learners to start with low-risk situations, and move gradually to more high-risk situations. Low-risk situations are either anonymous evaluations, or expressing formative feedback to a peer. Here are some examples, where learners can develop the ability to express honest opinions:

Rate-the-Argument: To an issue posed in a group, have everyone decide their opinion then write down two arguments to support that opinion. Each argument is written on a separate yellow "stickie". Post the stickies on flipcharts. Now give everyone a marker pen or gummed coloured dots. They circulate around the flipcharts, reading the arguments. Each person marks the ‘best’ four arguments, using personal opinion, and jots their reason for choosing an argument to be a ‘best’. Groups re-congregate and examine which "stickies" received the most marks as ‘best’. Together discuss the criteria used to assess these arguments.

Ink-Shedding: Newman (1991) suggests you ask people to ‘free-write’ for ten or fifteen minutes , scribbling down honest ideas about a relevant topic. Depending on the situation, for example, people can write their arguments for a particular issue, their assessment of a problem, their recommendations for a solution, their response to an experience, and so on. At the end of ten minutes, have everyone pass their writing to someone else in the group, without signing it. That person reads the page, then ‘freewrites’ a personal response. The responses note areas where the readers agree or disagree, ideas that they appreciate as original, and ideas they would challenge for whatever reason. Papers are passed on to one more reader, who reads both the original and the first response, then writes a second response. The focus is always on honest personal response, and no one signs their names. After people are given time to re-read their original writing and the two responses, discuss insights and other learnings.

Writing Conferences: Have learners in pairs respond to each other’s written drafts of assignments or reports. First show people what to give feedback on. (See sample below: Peer Responses in One-on-One Writing Conference). Ask writers to state a few things for which they desire feedback. The responder restricts feedback to personal response: I liked this part because . . . I found this part unclear because . . .

Group Self-Assessment: Have people who have just completed a group activity each jot down their honest, private assessment of how the group worked. Provide a simple list of criteria before they write these assessments. Then invite the group to discuss together their individual assessments, noting areas of discrepancy and talking these through. A sample form for group assessment is provided here.

• High-risk situations of peer assessment are usually face-to-face and/or involve summative evaluation. The most difficult for many may be delivering ‘bad news’ to a person whose confidence is already low, who is easily hurt, or who becomes defensive.
  
  
• Expressing feedback to a peer should always be a dialogue, sharing meanings about what happened. As Rough (1994) shows very clearly, measurement in dynamic environments must be flexible, emergent, and work from multiple perspectives. Expression should be straightforward and matter-of-fact. Feedback should focus on the behaviours observed and the observer’s experience of those behaviors. Check perceptions frequently with the one who was observed. Where there is discrepancy between observer’s and the observed’s perceptions or interpretations, explore it together. There is no ‘right’ or ‘wrong’ perception, only different windows on an experience which can shed light for one another.
  
  
• Help people develop a method for expressing feedback after observing a peer in an activity. Here is one suggestion:
  • First ask the other how the experience went for him or her. What did this person notice most? What would he or she most like feedback about? Listen carefully.
  • Then explain to the other the experience as you understood it: what happened and how you reacted to what happened. Tell what you were looking for. As much as possible, explain ‘where you’re coming from’: the values and assumptions through which you observed and interpreted.
  • Start and end with positive observations, actions and words the observer appreciated.
  • To express critical feedback, stick to what you observed or what you were looking for but did not see. For example, "I did not see/hear you_____", or "I saw/heard you_____", when Use the benchmarks of the criteria.
    
    
• Here are some suggested activities for easing from low-risk to more high-risk situations of peer assessment:

Bouquets and Beefs: Peers write a note, signing their name, providing personal response to a peer’s performance. One thing I appreciate . . . One thing I learned . . . One thing I wish had happened . . . Emphasize the need for specific description. General, vague responses are not helpful to a peer.

Practice Giving Critical Feedback: View a videoclip together of an unsuccessful performance. Note your responses/what you observed/your judgments (using simple criteria). Discuss in pairs, as well as strategies for delivering feedback. Make new pairs and practice delivering the critical feedback to the new partner. Partner provides a personal response to the peer assessor: his or her feelings during the feedback, and further actions he or she might take.

4. Learning how to hear, interpret, and act on feedback provided by others

• Learners may need reminding that peer assessment is only one small piece of data available to them as they move towards deeper self-knowledge. To discount the feedback is to refuse the gift of valuable information.
  
  
• Some learners assume that assessment is either right or wrong. They may need reminding that all judgments are framed by a particular limited perspective (their own included), and all perspectives help illuminate the truth.
  
  
• Learners may have difficulty hearing assessment offered by their peers. Some seize upon and magnify the critical comments, ignoring the appreciative observations. Others hear only those comments which resonate with their own opinions. Encourage learners to take notes as they receive feedback, jotting down everything they hear then reviewing these notes later. As they listen, invite learners to ask questions and to paraphrase what they think they are hearing, to clarify the feedback.
  
  
• Remember, a person can choose to ‘own’ or not own whatever parts of the feedback make most sense to them. It’s ultimately up to the learner to decide after a feedback session which observations make most sense to them.
  
  
• Both the observer and the observed in a peer assessment need to realize that it’s okay to leave discrepant perceptions ‘unresolved’. When one person interprets an experience differently than another, allow both perspectives to exist alongside one another rather than insisting, outwardly or inwardly, ‘I know I’m right and you’re wrong’.
  
  
• Invite learners to focus on what they can learn from the feedback, not what others think of them. Learners might ask themselves these five questions when receiving peer assessment:
  • Did anything make you feel defensive or hurt? why?
  • What did this feedback confirm, that you already know or suspect?
  • What new insights and lessons were opened for you by this feedback?
  • What responses surprised you? Ask yourself why , and what you can learn from it.
  • What one thing will you act on? what will you do first to improve it?

Benefits of Peer Assessment

If you have difficulty ‘selling’ the idea of peer assessment to learners, managers, clients, colleagues, institutional gatekeepers, or other stakeholders in evaluation, here is a list of the benefits that a system of peer assessment can offer. The assumption is that peer assessment is a complement to other forms of evaluation, including self assessment.

1. People sharpen their ability to observe both critically and appreciatively their own and others’ thoughts and actions.
   
   
2. People become more aware of their own implicit personal biases. They begin to see how these biases influence their judging processes, both when they assess themselves and when they respond to others.
   
   
3. People are opened to a wide variety of perspectives and observational styles in judging.
   
   
4. People develop ability to judge what they observe more by using particular standards and criteria, and less by personal preference and emotional appeal.
   
   
5. People become aware of complex issues in establishing defensible criteria for judging quality of work and thought.
   
   
6. People learn how to express their evaluative opinions in ways that do not offend.
   
   
7. People gain confidence and trust that presenting honest feedback to someone else will not necessarily be a negative experience. Thus, people learn to provide constructive feedback to one another as a regular routine.
   
   
8. People learn how to accept constructive feedback from others, without viewing such assessment as a personal attack.

Potential Concerns of Peer Assessment

For the record, we have assembled here a list of the drawbacks and potential pitfalls of peer assessment. Where possible we have indicated some preventive measures or ‘solutions’.

1. There are definitely risks in peer assessment. Some adults need to fine-tune their skills in diplomacy, and others need to learn accept feedback without viewing it as personal attack.
   
   
2. Humiliation can occur if the dignity of a learner is not preserved throughout the assessment process. To avoid this, frame peer assessment within the framework of a caring, appreciative relationship. Emphasize the need to stick with observable behaviors in the here-and-now, and personal responses to these observations. Communication of feedback unfolds in a private dialogue, never in a public environment. And finally, peer assessment is understood to be confidential, between the observers, the observed, and the instructor where there is one.
   
   
3. Power is always an issue. In an assessment situation, power accrues to the evaluator. A learner is in effect being asked to temporarily surrender to a peer as an object of scrutiny. This dynamic needs to be acknowledged openly and understood by all participants as a voluntary and temporary relation. It can be mitigated when the evaluator affirms his or her peer, offering feedback within the context of caring for a peer as a "critical friend". Both observed and observer enter assessment as an opportunity to dialogue about different experiences and perspectives, to learn from one another.
   
   
4. Sometimes peers are afraid to be honest, perhaps because our society attaches many negative connotations to constructive criticism as being ‘overly critical’ and ‘judgmental’. The "critical friend" approach can help. Clear criteria and specific descriptions of indicators can help people learn to observe with attention to detail, and describe with precise phrases. This removes the focus from the ‘personal’, placing it on sharp observations and careful, valid interpretations of behaviours.
   
   
5. Communication styles among people are very different. Tannen (1990) describes two main styles: indirect and direct. Direct communicators may appear abrupt and even rude to indirect speakers, while the indirect style uses so much nuance that a direct person doesn’t ‘get it’. People need to be aware of their own communication style and how it’s received and possibly misinterpreted by others. They also need to be resilient and allow elbow room for other communication styles. Check perceptions and interpretations!
   
   
6. Peers may provide different feedback to instructors. This can be problematic if peer assessment is an important part of summative assessment for a program. Our contention here is that this problem should not emerge if learners have clear criteria (either provided or developed by themselves), models and specific descriptive indicators to help them visualize what these criteria look like, and practice in applying the criteria. The relationship dynamics of offering critical feedback in a particularly must also be recognized openly. If all of these elements are in place and there still are significant discrepancies between instructor and peer assessment, these should be discussed. It may be that instructors are not applying the same criteria in the same way as the learners. Or, there may be relational issues interfering with simple comparison of criteria to observations of peer performance.

Overall . . .

We are strong believers that the advantages of peer assessment outweigh the potential disadvantages. The key, we believe, is to help learners develop the skills of observing their peers and interpreting these observations, then communicating their perceptions with compassion and respect. Distinguish carefully between personal response and criteria ratings. Be absolutely clear which kind of feedback is requested in a particular situation, and help people develop the skills of preparing and presenting the feedback with specific description. We suggest the notion of "critical friend" is helpful to encourage peer assessment as a positive learning experience for all participants.

References

Argyris, C. (1993).Overcoming organizational defences. Needham, MA: Allyn and Bacon.

Black, L. C. (1993). Portfolio assessment. In T. W. Banta and Associates (Eds.), Making a difference: Outcomes of a decade of assessment in higher education. San Francisco: Jossey Bass.

Congor, J. (1996). Can we really train leadership? Strategy and Business, Winter, 1996, 2-15.

Davie, L. E. (1997). Program evaluation for adults. In T. Barer-Stein and J. A. Draper (Eds.), The craft of teaching adults. Toronto: Culture Concepts.

Goleman, D. (1998).Working with emotional intelligence. New York: Bantam Books.

Heimlich, J. & Norland, E. (1994). Developing teaching style in adult education. San Francisco: Jossey-Bass.

Marienau, C. (1994). Self-assessment: An essential skill for learning and performance in the workplace. In Proceedings of the 35th Annual Adult Education Research Conference.. University of Tennessee, Knoxville, Tennessee.

Newman, J. (1991).Interwoven conversations: Learning and teaching through critical reflection.. Toronto: Ontario Institute for Studies in Education Press.

Palmer, P. J. (1998). The grace of great things: Reclaiming the sacred in knowing, teaching, and learning. The Sun, September, 1998, 24-28.

Rough, J. (1994). Measuring training from a new science perspective. Journal for Quality and Participation , October/November 1994, 12-16.

Schon, D. A. (1987). Educating the reflective practitioner: Toward a new design for learning and teaching in the professions. San Francisco: Jossey-Bass.

Senge, P. (1999). The dance of change: The challenges of sustaining momentum in the learning organization. New York: Doubleday.

Tannen, D. (1990). You just don’t understand: Women and men in conversation. New York: Ballantine Books.

Wheatley, M. J. (1994). Leadership and the new science: Learning about organizations from an orderly universe. San Francisco: Berrett: Koehler.

Note: This article is an excerpt from the book The Art of Evaluation: A Handbook for Educators and Trainers, published in 1999 by Thompson Educational Publishers, Toronto, Ontario.