FAQs about Assessment

An answer to this question and more about preventing misconduct, plagiarism, code of student behaviour, academic integrity in exams and additional resources can be found on our Academic Integrity page.

At the moment, IST is working on releasing Smart Exam Monitor (SEM) which can be used as a proctoring service within the Chrome browser. We suggest using SEM along with another eClass plug-in called Examlock. Examlock is a free plug-in that prevents students from being able to navigate away from the exam while they are taking it. It also records the student's screen, so if they are able to navigate away, the instructor will be notified.

Unfortunately, no method of delivering online quizzes is 100% secure. We encourage instructors to consider using multiple versions of auto-graded questions and replacing 30% of their questions each term the course is offered

That being said, instructors are encouraged to consider alternatives to closed-book exams. For more information, see Section 5 of our Teaching & Learning Online Course.

At the moment, IST is working on releasing Smart Exam Monitor (SEM) which can be used as a proctoring service within the Chrome browser. We suggest using SEM along with another eClass plug-in called Examlock. Examlock is a free plug-in that prevents students from being able to navigate away from the exam while they are taking it. It also records the student's screen, so if they are able to navigate away, the instructor will be notified.

That being said, instructors are encouraged to consider alternatives to closed-book exams. For more information, see Section 5 of our Teaching & Learning Online Course.

This article titled "Designing engaging assessment through the use of social media and collaborative technologies" by Richard Walker and Martin Jenkins may answer some of your questions.

Reference
Walker, R., & Jenkins, M. (2019). Designing engaging assessment through the use of social media and collaborative technologies. In Bryan, C. & Clegg, K. (Eds.) Innovative Assessment in Higher Education: A Handbook for Academic Practitioners (pp. 163 - 173). Routledge.

When deciding how much time to allow for an online multiple-choice exam you need to consider the length of your exam questions and the complexity or difficulty of your questions. A common estimate for multiple-choice questions is approximately 60 seconds (1 minute) per question. However, student performance may be hampered by high levels of anxiety and response times might need to be longer when items are more difficult, contain more words, pictures, or response items (Cox, 2019). Instead, the multiple-choice test should allow 75 to 90 seconds (1.25 to 1.5 minutes) per question to provide more students with an opportunity to answer all of the questions (2019).

The literature also suggests that multiple-choice questions should be simple in format (e.g., avoid use of complex item types or answering procedures), challenging but also allow students to succeed often, target specific cognitive processes that correspond to learning objectives, and use three plausible response options (Cox, 2019; Butler, 2018; Tarrant & Ware, 2009). Three-option items should be the standard on multiple-choice tests because (2009): a) three-option items perform equally as well as four-option items, b) require less time to develop and administer, and c) additional options often do not improve test reliability and validity (offer no psychometric advantage). Finally, the literature also indicates that providing feedback after a multiple-choice test enhances its positive effects on learning and substantially reduces or eliminates its negative effects (2019).

Works Cited
Butler, A. C. (2018). Multiple-choice testing in education: Are the best practices for assessment also good for learning? Journal of Applied Research in Memory and Cognition, 7(3), 323-331.

Cox, C. W. (2019). Best Practice Tips for the Assessment of Learning of Undergraduate Nursing Students via Multiple-Choice Questions. Nursing education perspectives, 40(4), 228-230.

Tarrant, M., & Ware, J. (2010). A comparison of the psychometric properties of three-and four-option multiple-choice questions in nursing assessments. Nurse education today, 30(6), 539-543.

Open-book exam (OBE) vs closed-book exam (CBE) comparison studies usually set the same time limit for both conditions. Michael et al. (2019) decided their total examination time based on one minute per question (60 questions, 60 minutes) for both cohorts (OBE and CBE) and found no statistically significant difference in mean examination scores. They also found reduced examination tension and stress for 70 per cent of the open-book exam students. Brightwell et al. (2004) required students to complete two assignments: a closed-book and an open-book examination, both with a time limit of one hour to allow one minute per question and 10 minutes revision time (12 seconds per question). Upon repeating the CBE test as OBE, students’ mean scores were not significantly different. Moreover, there was also no significant difference between the mean score of those who completed the open book assessment within the time limit (85% of students) and those requiring more time (15% of students requiring on average 1 minute 37 seconds additional time with a minimum of a few seconds to a maximum of 10 minutes).

In a systemic review (Durning et al., 2016) in three articles examinees took 10% to 60% longer to complete open-book exams (Agarwal & Roediger, 2011; Brightwell et al., 2004; Weber et al., 1983). Taking longer to complete OBEs could either influence the test’s reliability, if testing time is kept constant, or influence the length of time that must be offered to candidates to complete an equally reliable exam. However, post-examination outcomes suggested little difference in testing effects or public perception between open-book and closed-book examinations (2016).

A second systemic review found advantages to each type (Johanns et al., 2019): closed-book exams seem to foster better recall and memorization skills, enhance students' test preparation and promote deeper learning, while open-book exams seem to be more engaging and challenge students to critically analyze and synthesize the material (2019). According to Myyry and Joutsenvirta (2015), this seems to be especially true for open-book, open-web essay-examinations where students are allowed to use study materials, such as books and notes, as well as other materials, such as Internet. The questions in this type of exams demanded higher order cognitive processing, such as applying and evaluating, in a way that students could not find responses from course material or other resources, but formulate the answers themselves. Students reported that these type of questions made responding a powerful learning experience, because they activated their thinking to combine and apply knowledge. About one-fifth claimed that answering questions in the online examination promoted deep learning and high-level information processing, and one third thought their learning represented more deep learning than surface learning. In this study (2015), students used approximately the same time to study for an online open-book open-web examination as for a closed-book examination, but over half of them reported using more time for responding and learning more from the online examination.

Works Cited
Agarwal, P.K., Roediger, H.L. 3rd. (2011). Expectancy of an open-book test decreases performance on a delayed closed-book test. Memory, 19, 836 – 852.

Brightwell, R., Daniel, J. H., & Stewart, A. (2004). Evaluation: Is an open book examination easier? Bioscience Education, 3(1), 1-10. doi: 10.3108/beej.2004.03000004.

Johanns, B., Dinkens, A., & Moore, J. (2017). A systematic review comparing open-book and closed-book examinations: Evaluating effects on development of critical thinking skills. Nurse education in practice, 27, 89-94.

Michael, K., Lyden, E., & Custer, T. (2019). Open-Book Examinations (OBEs) in an Ultrasound Physics Course: A Good Idea or a Bad Experiment?. Journal of Diagnostic Medical Sonography, 35(3), 174-180.

Myyry, L., & Joutsenvirta, T. (2015). Open-book, open-web online examinations: Developing examination practices to support university students’ learning and self-efficacy. Active Learning in Higher Education, 16(2), 119-132.

Tarrant, M., & Ware, J. (2010). A comparison of the psychometric properties of three-and four-option multiple-choice questions in nursing assessments. Nurse education today, 30(6), 539-543.

Weber, L.J., McBee, J.K., Krebs, J.E. (1983). Take home tests: An experimental study. Res High Educ. 18, 473 – 483.

An answer to this question can be found on our Alternatives to Traditional Exams document.

Consider asking the student which form of communication (Zoom, Google hangouts, or telephone) they would prefer for a conversation about the student's learning. While chat and email allow this conversation to be documented, the risks for misinterpretation and escalation are very high. Express your concern for the student's learning in the course and invite the student to a discussion about how the student's learning experience can change. During the discussion, communicate to the student the observable behaviours you are concerned about and ask the student why these behaviours are occurring. Take it from here.

Sometimes the behaviours are about concerns outside of school. Sometimes it is a small change to teaching and learning environment that can have a significant impact. Sometimes it is an issue between students. Listen carefully to the student and don't be afraid to say you'd like a day or two to think about your reply if you are uncertain how to respond at the moment.

Individual feedback is important for students throughout their learning experiences. If you would prefer to have a meeting to give feedback to your students, and you have the time to do this, it is a great option since it allows you to have a conversation with the student.

Another alternative to writing feedback reports or individual meetings would be to provide feedback to students through short audio or video recordings shared privately through Google Drive.

In his book, Engaging ideas: the professor's guide to integrating writing, critical thinking, and active learning in the classroom, 2nd ed. John Bean (2011), provides several strategies to collect writing work from students so that instructors can have some feedback on students' writing progress. Ideally, instructors collecting work to check on students' progress can collect work as often as possible. R. Graves, professor emeritus and former director of the Writing Assessment Centre, says that he never went more than a week or two without seeing students' work so that he could adjust as necessary. One strategy you might employ is to have students post a quick summary of the reading so that you have some idea of what they are taking from the readings and how they are writing.

Additionally, you might want to consider giving feedback beyond individually, written feedback and these articles offer some potential avenues to explore:

• Consider using group and personalized feedback: https://onlinelibrary.wiley.com/doi/full/10.1111/bjet.12592
• Consider multiple modes of feedback:
  https://onlinelibrary-wiley-com.login.ezproxy.library.ualberta.ca/doi/full/10.1111/bjet.12749

Remember that not everything needs to be graded. Sometimes, peer-assessment or self-assessment can provide just as much feedback as instructor-graded materials. For more on this, see our descriptions of summative and formative assessment in "Section 5: Assessing learning in an online course" in our Teaching & Learning Online course.

So, what does need to be graded in a course? Instructors should have enough graded opportunities that students are able to show achievement of the course learning outcomes in several different formats. For those pieces of a course that are being summatively assessed, especially when auto-graded questions are not an effective option, instructor marking load can be lightened by using well-defined rubrics.

Student presentations can be completed either asynchronously or synchronously.

Asynchronously, student presentations could be completed using discussion forums.

• Have each group create a video of their presentation (they can record with Google Meet or Zoom) and embed the video onto a discussion forum.
• Questions can be posed by peers, the instructor, or stakeholders through the eClass forum and answered by the members of the group.
  • It is suggested that instructors coach students on evenly distributing their responses to these questions.

Synchronously, student presentations could be completed using either Google Meet or Zoom.

• The instructor can allow the student to share their screen if the presentation requires a presentation.
• If the presentation requires small group discussion, the instructor can either act as the 'breakout room coordinator' for the presenters (creating and opening breakout rooms) or make the student the host of the room during the presentation (note: you can reclaim the host at any time during the meeting as the meeting creator).
• Questions can be asked and answered through zoom using either the chat or audio/video (note, the host can enable/disable the chat at any point during the meeting).
  • We recommend disabling the chat during student presentations since it can be very distracting while one is presenting. However, if students are in groups, they could be coached to have one person monitoring and responding in the chat throughout the presentation; this might be a chance for those students who have difficulty orally presenting work to shine.

Answers to this question and more can be found on our Assessing Group Work and Participation page.