Assessing Adult Learning Using Portfolios

by Tara J. Fenwick, Asst. Professor
Department of Educational Policy Studies
University of Alberta, Edmonton, Alberta, CANADA T6G 2G5
tara.fenwick@ualberta.ca

Paper presented at the Atlantic Teaching Showcase Conference, October 1996, Universite St. Anne’s, Nova Scotia

(also available in The Art of Evaluation: A Handbook for Educators and Trainers by Tara Fenwick and Jim Parsons, published by Thompson Educational Publishers, Toronto, 2000.)

Interest is growing in portfolios as a form of assessment that can provide a more authentic, student-centered experience and growth-oriented interactions between learners and instructors than tests and final assignments. Portfolios have been explored for use in a variety of instructional contexts, including elementary schooling (Lamme and Hysmith, 1991; Hebert, 1992), literacy (Valencia and Calfee, 1991), college programs in diverse disciplines (Valeri-Gold, Olson, and Deming, 1992; Black, 1993), developing youth employability skills (Stemmer, Brown and Smith, 1992), pre-service professional preparation programs(Winsor and Ellefson, 1995) and evaluation of professional performance (Bosetti, 1995). There are many benefits in using portfolios to assess adult learning, but there are some disadvantages as well. Careful planning, patience and flexibility are essential throughout the process of working through portfolios with students. The purpose of this paper is to provide an introductory overview of the nature of portfolio assessment, showing examples, describing practical details, and offering suggestions for implementation.

What is portfolio assessment?

A portfolio is a folder assembled by the learner, containing samples of the learner’s work collected over a short period of time such as during one particular course of instruction, or a longer period such as the two years of a student’s post-secondary program. The kinds of items will reflect the nature of the course content. An art portfolio may contain sketches, photographs, and paintings in various stages of completion. A portfolio for a management development program may contain written case studies describing aspects of the learner’s workplace situation, a leadership style inventory, personal written reflections completed at various points, a plan for implementing a new program, an analysis of a particular management problem, a videotape showing a presentation or a chairing of a meeting. Any of the following items are commonly collected in learner portfolios:

1. Learners’ formal written papers, articles, descriptions, case studies, problem solutions.
2. Assessment inventories (such as personality or learning style inventories)
3. Photographs or drawings of learner-created products, artwork
4. Videotapes of learner actions or presentations
5. Learner journal, memos, or personal responses
6. Written reports/attestations to learner performance by others (peers and colleagues, or supervisors) about learner’s behavior
7. Test papers and results
8. Written observations of learner performance contributed by the instructor

Portfolios are collected and used in different ways according to the instructional purpose. Black (1993) describes portfolios used extensively at Emporia State University (ESU) to assess students’ basic skills, knowledge, and values that are difficult to evaluate: integration, critical thinking, analysis of issues, values clarification, multicultural issues, etc. A limited number of items was chosen to avoid voluminous folders to be assessed, and because staff felt that much information could be learned from relatively few items. Students submitted nine products. Regularly scheduled reviews of the portfolios between instructor and student took place at key points in the student’s academic career. The process of developing portfolio required discussion among staff of program goals and agreement to develop a statement of what students were to learn. A meeting of general education faculty and interested students was held to communicate this information. Course instructors then explained to students the intentions of course, how it fit into general education program, and how it helped achieve the broad goals of the program. Curriculum modifications were made as a result of portfolios, and the staff went on to develop scoring protocols with descriptors, and to explore the use of electronic portfolios.

At Manhattanville College, students submitted a large brown envelope twice during the academic career to demonstrate specified competencies such as research ability, critical skills and interpretation (Black, 1993). At Western College (Miami University) staff first summarized portfolios by types: papers, lab reports, projects, in-class writing, exams, quizzes, and computer exercises, then broke down the categories further (i.e. papers - research, essay, reaction, creative, journal, etc.). The percentage of assignment in relation to the total work was calculated using page numbers. Staff then reviewed portfolios together in a workshop setting, finding exemplar pieces (i.e. that gave evidence of an important goal like critical thinking). When the focus shifted to finding out what students were doing rather than how well, staff reported that defensiveness was reduced. The workshop review helped each staff member know what to look for when assessing portfolios in individual classes (Black, 1993).

How are portfolios actually used?

Developing a portfolio approach is challenging, and refining the program takes time. There are no prescriptions that transfer easily from one context to another, nor is there any one best way. Each staff or individual who decides to use a portfolio approach must develop unique expectations, guidelines for content, and criteria for assessing suitable for the purpose of the course and the objectives of involving students in the portfolio activity itself. Samples of student work can be used to establish benchmarks for scoring, and to generate descriptors and numeric grades for each level of scoring. Some instructors have found that if they do not attach numeric grades to the portfolios, the institution does not grant credibility to the portfolio as a valid assessment tool.

Here are a suggested series of steps to follow when beginning to use portfolios:

1. Decide first: What is to be collected, by whom, and when? Where are the portfolios to be kept? How are disparate items to be evaluated fairly and with reasonable validity? Then be prepared to modify your initial plans as you and the students explore possibilities and impossibilities of using portfolios. Experienced users offer two suggestions for initial planning of portfolio use: 1) Limit the number of products required in a portfolio; and 2) Use portfolios in conjunction with other methods of assessment: surveys, standardized tests, and formally graded assignments.
   
   
2. Spend time introducing portfolios to learners: Experienced portfolio-users suggest that you give as much help to learners up front as possible. Be as clear as you can about what things can go in a portfolio, being careful to balance your directions with the essential theme of learners’ freedom to create and to choose from their creations. If possible, show samples. Share with learners the specific benefits of the portfolio approach. Help learners know what to expect in the emergent portfolio process. Feelings of ambiguity and a sense of unclear goals are natural, and everyone needs to be open to the trial-and-learn process of building portfolios. Show them your own intent to adapt your expectations as the portfolios grow.
   
   
3. Work with learners throughout the portfolio process: Communicate to learners your general expectations for the portfolio contents, then work with them throughout the period of collecting. Learners need lots of help focusing and selecting, especially if they are unused to the portfolio method of assessment. Most simply need confirmation that the items they are creating and selecting for their portfolios do indeed meet whatever might be your minimum expectations. You can help by providing time for learners to share their portfolios with one another and talk together about the process of collecting items.
   
   
4. Hold periodic conferences with learners: Some programs build in two formal portfolio conferences between instructor and each learner, one conference mid-way through the program and the second at the end of the program. At the conference, the learner leads the instructor through the portfolio or a section of the portfolio. The learner discusses the samples of work chosen, perhaps the background of the items and reasons for their choice, and describes an assessment of his or her learning in the areas demonstrated by the portfolio.
   
   
5. Grade portfolios holistically: It is difficult to grade each item in the portfolio, then add up the marks for a final grade. Different program contents dictate different needs for items in the portfolio. First, items in one student’s portfolio often differ in kind and number from items in another portfolio so an item-by-item grading process will be counter-productive. Second, such a grading process will be very time-consuming. Third, the point of the portfolio is to demonstrate overall growth in learning. Item-by-item grading assesses early items against the same sorts of external criteria used to measure final items. Instead, try a grading system that allows a holistic mark blending different aspects of the portfolio process: grading of particular items in the portfolio (perhaps the learners’ choice of the best), kinds of learning growth demonstrated, evidence of the learners’ insight and self-assessment of their own learning process as evidenced in the conferences, and balance and range of skills represented in the portfolio items. As you see the kinds of portfolios that are developing (and allow students to share these with one another) work out, ideally with the learners, the criteria for assessment and the sorts of indicators that will be most helpful. The process is very much an emergent one. In some institutions, instructors meet to work out general guidelines for the contents and assessment criteria of student portfolios; specific details are worked out in each class or with each learner. A sample format developed to grade portfolios of learners in a professional training program is shown in Figure 1.
   
   
6. Spend time responding to learners: Either orally or in writing, provide learners with thoughtful responses to their assembled portfolios and the self-evaluation evident in their rationalizations for items chosen. Feedback throughout the process of collecting items for portfolios is essential to help learners focus and clarify the purpose of the portfolio in their own learning.

What are the benefits of using portfolios?

Portfolios show learner growth over time. They work well to follow a student through a program from course to course and year to year, providing continuity, integration, and a record of overall progress.

Portfolios involve learners directly in their own learning and evaluation. When learners can exercise more control over the kinds of learning products that are evaluated, their sense of responsibility and ownership for the learning increases. The act of creating a portfolio also helps move learners towards self-evaluation.

Portfolios show the process of learning, not just the outcomes. Samples are collected at various periods throughout the learning process, thus reflecting the learners’ incremental stages of development. The transitions and growth that become apparent are fascinating for both learner and teacher. Learners find out more about their own learning process. Besides, people’s motivation for following through any project like learning usually increases when they can see visible evidence of their progress accumulating.

Portfolios build learner confidence. One of the most common reports from instructors using portfolios is the powerful effect they have on student self-esteem. The accumulated folder of concrete, personal accomplishments is visible proof to students of their own ability and mastery of the new learning.

Portfolios are holistic measures of learning. They contain diverse samples of work that demonstrate a variety of interconnected learner knowledge and skills. Thus portfolios reflect more authentically the capability and achievements of learners.

Portfolios are a learning experience in themselves. To assemble a portfolio, learners must reflect on their learning, evaluate their own learning products, make selections representing their own competencies and rationalize their choices. This process is a powerful way to raise student’s awareness of the learning process. As the portfolio grows, its very existence begins to shape the learner’s goals. Meanwhile the student’s insights about the learning process, revealed through the selection and reflection involved in assembling the portfolio, continue to shape the student’s approaches to learning experiences and assignments.

Portfolios are useful for learners seeking employment. The materials in a portfolio can be easily translated into a resume. Some employers now require applicants to provide portfolios of their work. Others can be persuaded by adult education instructors and institutions of the advantages of portfolios in the hiring process. Pamela Winsor and Bryan Ellefson of Lethbridge, Alberta, for example, convinced local school district superintendents to accept portfolios from student teachers applying for teaching jobs (Winsor and Ellefson, 1995).

Portfolios are useful indicators of gaps in the program. At Miami University in Ohio, portfolio assessment revealed that students had little skill in using the library and conducting research (Black, 1993). Faculty obtained resources and made a commitment to increasing instruction in these areas. At Emporia State University, student portfolios demonstrated that they weren’t integrating knowledge of different discipline areas. Key courses were refined to emphasize individual responses, connections between areas, and integration of the discipline with society (Black, 1993).

Portfolios are an effective means of faculty development. Portfolios require staffs to work together to develop criteria and scoring tools, agree on items for inclusion, and make changes to programs and instruction based on portfolio results. Developing a portfolio approach requires staff to reexamine instructional goals and communicate these clearly to one another and to students. Because portfolios reach across courses, they promote collaborative teaching.

What are some potential concerns in using portfolios?

1. Some instructors wonder how a portfolio can be a valid or reliable measure of learner achievement. Constructs like "validity" and "reliability" assume that learning outcomes should be pre-determined and measurable against an external standard. Portfolios assume that learning unfolds in different ways, with different outcomes, for each learner. Portfolios allow learners to reveal skills, creative abilities and understandings that are unique, and possibly not anticipated by the instructor. In programs and courses where certain performance standards must be pre-determined and met exactly, portfolios should be supplemented with other kinds of assessment tools (such as testing and students demonstration of a particular skill).
   
   
2. The learner may choose samples of work that reflect different outcomes and performance standards than what have been pre-determined for the learner to achieve.
   For instance, in the case of trade or professional accreditation, particular minimal standards of achievement must be demonstrated. The learner’s own creative products reflecting a particular direction of learning may be admirable, but not always helpful indicators of the extent to which the learner has achieved the minimal standards.
   
   
3. Portfolios are often time-consuming for instructors to evaluate. This issue cannot be considered outside of context. Some instructors spend far more time writing comments on learners’ final essays than they would in grading a portfolio of products holistically. Some instructors find that time spend conferencing with students about their portfolios is valuable instructional time, and cannot be considered as "extra" marking time. Learning how to use any unfamiliar assessment approach will be, in the beginning, more time-consuming than customary approaches.
   
   
4. Portfolios contain disparate collections of items, which make evaluation a slippery and sometimes ambiguous process. The more wide-open and flexible the specifications for the learners’ portfolios, the more thought must be given to ways of fairly evaluating different students’ work. How can a portfolio containing several pieces of personal reflective writing, one containing one professionally published article, and one containing a few business correspondences be compared as final submissions in a college writing course?
   
   
5. Portfolios can provide a history of learning and growth, a structured record of learner accomplishment, a vehicle to engage learners in active reflection on their experience, a way to develop self-evaluation habits and skill, as well as a method for assessing progress.
   
   
6. Building a portfolio involves point-in-time performance assessment. The construction of a portfolio allows both the students and the instructor to assess transitional change as well as cumulative learning.
   
   
7. Students are sometimes skeptical at first about the whole idea of portfolios. They often don’t know what to do, and wonder about the time involved. Some are unused to self-evaluation, and are uncomfortable with the decision-making process required. Some find it difficult to complete products that are not "handed in" as an "assignment" for an immediate assessment by an external authority. Give them lots of time to experiment and become comfortable with the responsibility they must assume in portfolio assessment. Provide supportive help, suggestions when asked, and feedback to early efforts as much as possible. Most of all, be patient and accepting as students struggle to find comfort

The following six strategies can help maximize the effectiveness of portfolio use to evaluate students in your program:

• define the purpose of the portfolio
• teach the students to self-reflect
• structure portfolio reviews by the students
• make time for peer evaluation
• share portfolios with students regularly
• give yourself time to get become comfortable with portfolio use

Conclusion

Portfolios offer a form of assessment that has been described as "authentic", grounded in activities connected with learners’ everyday problems, insights, and practical dilemmas. Portfolios demand learner responsibility in the process of creating, reflecting on and evaluating learning, and portfolios focus on the learning process as well as its outcomes. However, the assembly of a portfolio represents for many adult learners a significant departure from the routines of instructor-designed and instructor-managed class assignments and tests that many have come to expect. Creating a portfolio requires new skills in selection, articulation and design, and a favorable attitude towards critical reflection and self-analysis. While learners usually can develop these skills during the process itself of building a portfolio, they often need assistance and patient understanding from course instructors as they work through the process.

Meanwhile for instructors, portfolio assessment poses difficult questions: What kinds of products demonstrate learning, and what kinds of learning are represented? How can these products be interpreted? How can a multitude of products representing different content in different forms for different purposes all be evaluated meaningfully? How can a numerical grade be assigned fairly? What information about the learner, the learner’s context and history, and the learner’s own meanings about the portfolio and what it represents of the learning process, are critical to understanding and evaluating the contents of a portfolio?

As those who have worked with adult learners through the portfolio process have attested, these questions are not impossible philosophical dilemmas. Portfolios are workable. Increasing examples and cases are emerging that demonstrate institutional solutions and instructors’ practical responses to the issues raised by using portfolios. In fact, these issues open dialogue among instructors and between instructors and learners about fundamental principles: what it means to learn, how learning can possibly be demonstrated and judged, and what is the role of an instructor in a learner’s reflective journey. Portfolio assessment, when planned and implemented thoughtfully, can promote learners’ personal and educational development, strengthen connections between course material and its application, and provide a focus for meaningful reflective conversation between instructors and students. It can focus both learners and instructors on their purposes, re-examining ‘What are we doing here?’ The decision to undertake portfolio assessment is a choice to forego control and management-by-measurement, and to embrace a more emergent dynamic method of evaluation that entwines judgment with the learning process itself.

Figure 1: Sample format for holistic grading of a portfolio. This format was developed to assess portfolios documenting the growth of people enrolled in a certificate program in Adult and Continuing Education. Their portfolios collected items showing their growth of skill and understanding in program planning, learning and development, foundations of education and facilitating adult education.

References

Arter, J. A., and Spandel, V. (1992). NCME Instructional Module: Using Portfolios of Student Work in Instruction and Assessment. Educational Measurement, 11 (1), 36-44.

Black, L.C. (1993). Portfolio assessment. In Banta, T. W. and Associates (Eds.), Making a difference: Outcomes of a decade of assessment in higher education.(pp. 139-150). San Francisco: Jossey-Bass.

Blake, J., Bachman, J., Frys, M.K., Holbert, P., Ivan, T., Sellitto, P. (1995). A portfolio-based assessment model for teachers: Encouraging professional growth. NASSP Bulletin, 79 (573), 37-46.

Bosetti, L. (1995). Teacher evaluation as pedagogic opportunity: A case for portfolio assessment. The Canadian Administrator, 35 (8), 1-12.

Hebert, E. A. (1992). Portfolios Invite Reflection from Students and Staff. Educational Leadership, 49, (8), 58-61.

Lamme, L. L., and Hysmith, C. (1991).One School's Adventure into Portfolio Assessment. Language Arts, 68 (8), 629-640.

Stemmer, P.; Brown, B.; and Smith, C. (1992). The Employability Skills Portfolio. Educational Leadership, 49 (6), 32-35.

Valencia, S. W., and Calfee, R. (1991). The Development and Use of Literacy Portfolios for Students, Classes, and Teachers. Applied Measurement in Education, 4 (4), 333-345.

Valeri-Gold, M.; Olson, J. R.; and Deming, M. P. (1992). Portfolios: Collaborative Authentic Assessment Opportunities for College Developmental Learners. Journal of Reading, 35 (4), 298-305.

Winsor, P. & Ellefson, B. (1995). Professional portfolios: Monitoring and documenting professional development of student teachers. Alberta English, 33 (1), 5-12.