• SAIL Info Session Launch Presentation (PDF)

• Rubric - Critical Thinking and Investigation (PDF)
• Rubric - Social Responsibility (PDF)
• Rubric - Lifelong Learning (PDF)
• Communication Rubric
• Rubric Workshop (Table 2.3)

• Data Collection Notice (PDF)

• Lifelong Learning Assessor Rating Sheet (PDF)
• Assessor Training (Table 3.1)

• Lifelong Learning Assessor Rating Sheet (PDF)
• Assessor Instructions (Table 4.1)

• Faculty Questionnaire (PDF)

• Sample Course Report - Lifelong Learning (PDF)

• N/A

• SAIL Coordinator Check List (PDF)

Carolyn

Alana

Co-Creation of Shared Institutional Rubrics

References

Course-Embedded Assignment Selection

Student Consent

Opt-in or Opt-out Student Consent

Random Selection from Consented Artifacts

References

References

Semi-Structured Debrief Prompts

Interpretation of Faculty Responses and Creation of Final Report

• organized responses by the three themes described in the Debrief questionnaire;
• summarized responses into broad statements by sorting for duplication, overlap, and synergies;

• sorted statements into 1) Strengths and 2) Challenges/Areas for Improvement;
• incorporated quantitative analysis of student consent rates, descriptive assessor ratings, and relevant contextual information that emerged during the pilot (e.g., internal and external environmental factors such as the global pandemic, participation rates of faculty in ILO Pod activities, time delays);
• drafted a summary report, including recommendations based on the SAIL Coordinators’ interpretation of the faculty responses in the context of the additional quantitative and qualitative data, and environmental factors;
• shared, via email, the summary report with faculty and provided an opportunity for faculty to edit, comment, and clarify findings using track changes;
• revised the report based on faculty feedback; and,
• initiated the Consultation phase.

• Course-level with a focus on using the results from a SAIL project to inform changes to course, assignment, and/or assessment design;
• Program- or departmental-level with a focus on improving program learning outcomes through an effective, ongoing, and regular system of assessment. This is greatly facilitated when programs have clearly articulated program learning outcomes and curriculum maps; and,
• Institutional level with a focus on documenting and demonstrating, at a high level, student achievement of ILOs.

Iterative Action Research Cycles

References

This focus means we can centre our attention on co-creating quality assurance initiatives that have educational value for faculty members, first and foremost.

References

Impact on Teaching Practice

References

Pilot #1: 2020-21

Pilot #2: 2021-2022

1. faculty engagement in ongoing reflection and curriculum enhancement; and,
2. measurement of student achievement of learning outcomes via a broadly scalable and generalizable process, regardless of program discipline.

Strengths

• is aligned with the principles for learning outcomes and assessment;
• provided deep faculty engagement in structured review through educational development;
• led to actionable outcomes with implications for course and assignment redesign;
• the ILO Pods persisted as peer support networks even after the pilot ended;
• the measurement of student achievement of institutional learning outcomes are based on multiple courses and representative of multiple disciplines; and,
• resulted in faculty knowledge mobilization related to the Scholarship of Teaching and Learning; including, a presentation at the 2021 IUPUI Assessment Institute and multiple publications thus contributing to the field of educational research.

Limitations

• is resource and time intensive in part due to the current manual-approach to gathering artifacts, distributing artifacts, and compiling reports – additional tools for collection and distribution should be explored;
• assessment can be challenging when assessing assignments outside one's discipline, particularly when the assignment is not completed in English (e.g., in a Japanese language course) or is highly technical (e.g., in a computing science course), or where the epistemology and is not captured in the assignment description;
• assumes that one course assignment can sufficiently provide opportunities for students to demonstrate achievement of an ILO; and,
• posed challenges for drawing conclusions at the course, program, and institutional level due to the variability in assessor ratings and range of course levels included without additional training and focus on inter-rater reliability.

References

• Dr. Jovan Groen, Director, Academic Quality, Western University
• Wendy Hall, Vice President Effectiveness and College Relations (Accreditation Liaison Officer / Public Relations Officer), Lower Columbia College
• Dr. Catharine Dishke Hondzel, Dean, Academic Excellence and Teaching Innovation, Lambton College
• Dr. Shannon Wagner, Vice President Research, Thompson Rivers University

Launch and Recruitment

Purpose of SAIL

SAIL responds to the call to "close the loop". 

Research-informed practices place faculty at the heart of

these quality improvement efforts.

Institutional Learning Outcomes

Principles for Learning Outcomes Assessment

References

• an overview of the SAIL methodology, including strengths and limitations, and the research findings;
• step-by-step instructions for implementing a SAIL pilot project at your educational institution;
• resources for quality assurance practitioners and educational developers to facilitate faculty learning and development, including rubric creation workshops and Assessor Training; 
• templates and supporting documents; and,
• articles for further reading.

• contextually-grounded;
• addresses real life problems; and,
• involves participating practitioners constructing meaning through reflection that leads to action.

Coordination

References

1. Growth and learning-oriented
2. Equitable and learner-centred
3. Faculty-driven
4. Ongoing cyclical improvement
5. Purposeful and holistic design
6. Reflexive approach to learning

Action Research

SAIL has always been and continues to be an iterative faculty-led process - hallmarks to SoTL research.

References

Territorial Acknowledgement

Institutional Context

Collegial Governance

Continuous Quality Improvement

References

• Voluntary engagement: Faculty are encouraged, but not required, to participate as this prioritizes the educational value of SAIL over a compliance-driven initiative.
• Immersive professional development: Faculty can commit a minimum of 35 hours during one semester to participate.
• Teaching-focused, collegial culture: To be successful, faculty must trust the process, their peers, and the SAIL Coordinators.
• Restricted to certain types of course-embedded assignments: Due to the use of a standardized rubric, SAIL is ideal for written and oral assignments, and poses challenges for some types of assignments, such as those that are primarily numerically-based (i.e., mathematics and statistics courses) and multiple choice exams.

Build Community and Reliability

Adapt Assessment of ILOs to Intended Purpose

Engage Students as Co-Investigators

References

• Faculty-led article was drafted and submitted for review; manuscript title "Strategic Assessment of Institutional Learning Outcomes: A Faculty-led Community of Practice Approach"

• LOA Report submitted to senior leadership and AoL Subcommittee for review and discussion
• SAIL 3.0 project proposal drafted, submitted to senior leadership (including funding proposal) and AoL Subcommittee for approval
• REB proposal amended and approved

• Launch SAIL 3.0
• Still pondering a few questions
  • How often should one undertake this type of initiative to obtain a meaningful image of student LO achievement? What could longitudinal data tell us?
    • Possible cycle of assessment is discussed in LOA Final Report (2021 and 2022); we recognize that there may be faculty interest to assess ILO achievement more frequently that may fall outside a SAIL structure but could apply the institutional rubrics and process detailed in this handbook internally by department or faculty/school
    • Longitudinal data could tell us the trends in student performance and provide feedback to instructors
    • Considerations to embed in faculty professional development or cyclical program review (optional)
  • Were there particular observations regarding disciplinary differences (in the rating results, or working with faculty from different backgrounds)?
    • Sample size (pilot phase)
  • What resource and commitment considerations are most important to note when setting up a program like this? And to sustain a program like this?
    • SAIL Coordinators (QA, EdDev) and project coordinator (admin) is very beneficial across the duration of the project, including submission of REB and privacy, and workshop/session facilitation
    • Governance (i.e., AoL Subcommittee)
    • Expenses (e.g., course release, event costs)