In Fall 2020, teaching and learning scholars at a regional university in Western Canada launched SAIL — the Strategic Assessment of Institutional Learning — an action research project investigating student achievement of institutional learning outcomes.
This Handbook provides a coordination-level view and practical design considerations for implementing a learning outcomes and assessment process in an educational setting. The Handbook is relevant to curriculum committees, higher education leaders, faculty members, quality assurance practitioners, and educational developers who are engaging in assessment of program and institutional learning outcomes.
Purpose of SAIL
More and more frequently, publicly-funded post-secondary institutions are called to justify their value by demonstrating evidence of student learning. Institutions collect a plethora of direct and indirect student learning data and are increasingly reporting out on assessment findings; however, they typically stop short of acting on the results gathered. Colloquially known as “closing the loop” or “closed-loop assessment” (Alstete, 1995; Maki, 2002), using assessment findings to improve student learning requires a well-articulated, detailed, and reflexive process. If successfully implemented, this process can increase the likelihood of improved student outcomes (Reich et al., 2019). Yet, few examples exist of institutions that use assessment findings to spur change and assess the impact of those changes on student learning (Banta & Blaich, 2010).
SAIL responds to the call to “close the loop”.
SAIL is driven by faculty members’ desire to further their students learning and educational institutions’ aspirations to engage faculty and students in ongoing formative assessment. Provincial regulations in Canada and institutional and programmatic accrediting bodies are increasingly requiring that post-secondary institutions engage in regular quality review and improvement processes.
Research-informed practices place faculty at the heart of these quality improvement efforts.
Faculty are called to collect, reflect on, and act as appropriate on meaningful data regarding student learning and student achievement of core competencies.
SAIL investigates methods for assessment of student learning to help faculty (and institutions) better adapt to current and changing needs of learners, while honouring disciplinary diversity and faculty autonomy over teaching and learning. SAIL is designed to encourage meaningful and actionable conversations about how to teach and assess institutional learning outcomes. Our research seeks to further educational excellence, knowledge-sharing, and reflective practices. SAIL contributes as a research project to the Scholarship of Teaching and Learning.
Institutional Learning Outcomes
Institutional learning outcomes (ILO) are direct statements that describe what students should know and be able to do upon graduation from a post-secondary institution. Many institutions articulate ‘signature’ learning outcomes that focus on transferable knowledge, skills, attitudes, values, and behaviours that can be evaluated and assessed.
When ILOs reflect the mission and vision of an institution and are explicitly linked to institutional, college, and departmental plans, they can support mission fulfilment. This is best supported when intentional efforts are made to embed ILOs into quality assurance processes, such as cyclical program review, strategic planning, and resource allocation; and is further enhanced when ILOs are incorporated into faculty learning and development opportunities, such as programming offered through centres of teaching and learning. For example, curricular mapping and scaffolding of course and program learning outcomes to institutional learning outcomes can illuminate insights into student learning. The information gathered can be used to support institution-wide initiatives and inform learning support planning and practices to continuously improve student learning.
Principles for Learning Outcomes Assessment
SAIL follows a principled-approach and is based on six principles for learning outcomes and assessment. These principles were collaboratively developed by faculty with engagement from students and, with appropriate consideration for local contextual factors, we suggest that the principles can act as guiding posts for any scholarship of teaching and learning project.
- Growth and learning-oriented
- Equitable and learner-centered
- Faculty-driven
- Ongoing cyclical improvement
- Purposeful and holistic design
- Reflexive approach to learning
These principles guide conversations and inform decision-making about learning outcomes and assessment throughout the study.
Collaborative Coordination
SAIL is collaboratively coordinated by quality assurance practitioners, educational developers, and faculty researchers with the overarching aim to improve student learning.
SAIL Pilot Projects
The SAIL approach to assessment of institutional learning outcomes is based on the use of a shared rubric and faculty peer-to-peer learning. The first three action research cycles consisted of the following elements:
- co-creation of a shared rubric to assess student achievement of an institutional learning outcome;
- identification of relevant student artifacts (course assignments);
- assessment of student artifacts using the shared rubric;
- review of a course report based on the assessments of two peers; and,
- feedback on the efficacy of the SAIL process.
Faculty are provided with a course report for their own use to reflect on and consider improvements to student learning. In addition, an aggregate report may be prepared based on the results of each ILO to inform institutional and departmental planning if sufficient comparable student data is gathered during the pilot.
Pilot #1 (2020 – 2021)
Pilot #2 (2021 – 2022)
The second iteration of SAIL involved two faculty-led ILO Pods aimed at assessing student achievement of Lifelong Learning and Social Responsibility during the Winter 2022 semester. Five disciplines were represented in the second study: social work, cooperative education, sociology, geography, and business.
Pilot #3 (2022 – 2023)
References
Alstete, J. W. (1995). Benchmarking in higher education: adapting best practices to improve quality. ASHE-ERIC Higher Education Report No. 5, Washington, DC, pp. 1-112.
Banta, T. W. & Blaich, C. (2010). Closing the assessment loop. Change: The Magazine of Higher Learning, 43(1), 22-27. https://doi.org/10.1080/00091383.2011.538642
Maki, P. L. (2002). Developing an assessment plan to learn about student learning. The Journal of Academic Librarianship, 28(21), 8-13.
Reich, A. Z., Collins, G.R., DeFranco, A.L. and Pieper, S.L. (2019). A recommended closed-loop assessment of learning outcomes process for hospitality programs: The experience of two programs, Part 1. International Hospitality Review, 33(1), 41-52. https://doi.org/10.1108/IHR-09-2018-0010
