Each faculty member engaging as a co-researcher with an ILO Pod is responsible for assessing two of their peers’ course assignments.
Faculty are provided a random sample of course assignments. The sample size can vary by ILO Pod. For example, an ILO Pod may collectively decide to assess 10 assignments, while another ILO Pod may choose to assess 15. Typically, ILO Pods opted for 10 random course assignments selected from each course, requiring that each faculty member assess 20 course assignments using the shared rubric.
Depending on the faculty members’ experience with using rubrics, their familiarity with the discipline they are assessing, and the relevance of the rubric to the course assignment design, it can take between 15 and 30 minutes to assess each assignment. In Pilots #1 and #2, faculty conducted the assessment during a time of their own choosing, rather than simultaneously with their peers, and submit their ratings by a pre-determined deadline. In Pilot #3, to address challenges with inter-rater reliability, peers assessed student assignments during a two-day assessment institute with multiple comparison checks (Austin et al., 2024; Hoare et al., forthcoming).
The following instructions are provided to faculty members to guide them through the assessment of selected course assignments (Table 4.1).
| Instructions | |
|---|---|
| 1. Review the rubric |
|
| 2. Review the assignment description |
|
| 3. Assess the assignments using the rubric and rating sheet provided |
|
| 4. Provide feedback on the rubric |
|
The Assessor Rating Sheet provides assessor ratings for 10 course assignments with each column representing a student, and each row the rating, a strength, and an area to further develop. Strength and Area to further develop descriptions are brief and often informed by the rubric descriptions. The fourth section of the Assessor Rating Sheet gathers insightful feedback that helps to inform Debrief discussions and improvements to SAIL.
References
Austin, L., Hoare, A., Thomas-Francois, K., Pypker, T., & Cao, L. A. N. (2024). The intersection of student assessment and faculty learning: Reflections on a method for assessing student achievement of institutional learning outcomes. Intersection: A Journal at the Intersection of Assessment and Learning, 5(2). https://aalhe.scholasticahq.com/article/122484-the-intersection-of-student-assessment-and-faculty-learning
Hoare, A., Austin, L., Thomas-Francois, K., & Pypker, T. (forthcoming). Student achievement of institutional learning outcomes: Case study of a regional university in Western Canada.
