Assessor Training is delivered by the SAIL Coordinators. Within the Coordination Team are trained faculty members. This team-based facilitation leverages the strengths of quality assurance practitioners who bring a “deep knowledge of internal and external educational policies and processes, project management skills, access to resources, and familiarity with curricular governance systems” (Hoare et al., 2022); and, educational developers and teaching faculty who bring knowledge of enhancement of teaching effectiveness, assessment design, research on teaching, teaching and learning theory and practice, facilitation skills, and understanding of collegial governance (Sharif, et al., 2019; Steinert, 2016; Wilcox, 1998).
Assessor Training is customized for each ILO Pod and is roughly two hours. Below are instructions for SAIL Coordinators to facilitate Assessor Training with participating faculty members (Table 3.1).
| Duration | Activity | Resources |
|---|---|---|
| 15 min. | Review the ILO Rubric with ILO Pod members. Are the categories and descriptions (rows) clear? Are the levels (columns) sufficiently distinct? | Rubric – Lifelong Learning (PDF), Rubric – Social Responsibility (PDF), Rubric – Critical Thinking and Investigation (PDF) |
| 15 min. | Review the sample assignment and description. Are all of the categories evident in the assignment description? Can the categories be assessed using the rubric? | Sample Assignment: A public (non-confidential) assignment sourced from institutionally posted examples online. |
| 15 min. | Review the ILO Rating Sheet. The Rating Sheet is used to track the results of multiple students depending on the sample size used for the SAIL project (e.g., n = 10). Note the level of achievement for each category in the rubric for each student. Indicate any categories that were not applicable with “N/A”. Provide a brief description of each student’s strengths (e.g., efficient strategies for seeking information). These strengths will be themed across the sample of students. Provide a brief description of an area to develop further (e.g., limited ambiguity addressed). These areas for further development will be themed across the sample of students. | Lifelong Learning Assessor Rating Sheet (PDF) |
| 30 min. | Practice using the forms. Individually, faculty practice using the ILO Rubric and Rating Sheet with the sample assignment provided. | Sample Assignment, ILO Rubric, ILO Rating Sheet |
| 30 min. | Within the ILO Pod, debrief the activity and provide feedback on the ILO Rubric. For each category, faculty members share the ratings that they chose for the sample assignment, as well as the strengths and areas for further development. Was there consensus in ratings among the ILO Pod? If not, discuss the different assessment ratings. Are any of the categories difficult to assess? What made it difficult? Are any of the categories clear or easy to assess? What made it easy? Are the levels easy/difficult to differentiate? Are there any changes that you would make to the rubric? | |
| 15 min. | Wrap up and discuss next steps. Remind the ILO Pod members of the deadlines for uploading the student artifacts to Moodle, for assessing their peers’ artifacts, and for submitting the completed ILO Rating Sheets for their peers. | Moodle Site (including where to upload ratings, templates and deadlines). |
Based on our experience delivering five Assessor Training sessions across two pilots, we suggest that future iterations allow for a longer training session with time to begin assessing course assignments and discuss initial ratings. This extended session was suggested by faculty as a way to further collegiality and improve inter-rater interpretation. Considerations for confidentiality about student artifacts and who contributed to the sample will need to be attended to as faculty would be able to recognize which students participated from their course. Suggestions for improving Assessor Training are discussed further under Future Considerations.
