Action Research Design prioritizing Qualitative Methods
SAIL methodology is based on an action research design, which has the dual benefit of generating practical solutions and empowering practitioners.
The process of action research involves engaging practitioners in systematic enquiry focused on generating solutions to practical problems and the subsequent development of activities to improve outcomes across multiple cycles (Koshy et al., 2010).
Quality action research, according to Levin and Greenwood (2001), is:
- addresses real life problems; and,
- involves participating practitioners constructing meaning through reflection that leads to action.
Each cycle of action research includes a plan-act-evaluate-reflect cycle. The cycle involves identifying the focus (e.g., a problem, changes for improvement, learning outcomes), developing a plan to assess progress, implementing the plan, reflecting on progress, and identifying necessary changes to improve and re-implement the plan thereby stimulating a new cycle (Koshy et al., 2010; Macintyre, 2000).
The SAIL Planning Cycle includes the application of qualitative methods (focus groups, rubric-based descriptive assessments, community consultations) with some initial quantitative descriptive analysis of consent rates. The eight-step SAIL Planning Cycle (Figure 1) is depicted below.
At Thompson Rivers University, the first and second iterations of SAIL involved the use of faculty-developed institutional rubrics and the collection of data to evaluate the extent to which students were achieving Critical Thinking and Investigation, Social Responsibility, and Lifelong Learning in courses designated as meeting these three institutional learning outcomes (ILO) – also referred to as ILO-approved courses. The data was gathered through faculty applying a shared rubric to their peers’ course-embedded assignments.
Once the faculty-led communities of practice (“ILO Pods”) are formed for each of the ILOs being assessed the following steps are undertaken:
|1. Preparation, Recruitment, and Launch
||Involves preparation, including consultation and approval, review of policies and compliance check (including Research Ethics Board), and recruitment of faculty members.|
|2. Shared Rubrics||Involves the development (or refinement) of shared institutional rubrics.|
|3. Student Consent and Artifacts||Involves the identification of appropriate student artifacts (course assignments) for assessment using the rubrics; and, notification to students of the consent process (e.g., opt-in or opt-out).|
|4. Assessor Training||Faculty participate in Assessor Training delivered by an educational developer and quality assurance practitioner.|
|5. Assessment of Student Learning||Faculty assess two of their peers’ assignments using the shared rubric and rating sheet.|
|6. Debrief||Faculty participate in a focus group and are guided through semi-structured prompts led by the SAIL Coordinators to determine the efficacy of the SAIL pilot project.|
|7. Institutional Consultation and Reporting||Findings and recommendations are drafted, reviewed, and disseminated.|
|8. Course (Re)Design Implications||Faculty review and discuss their peers’ assessment and feedback of the evidence provided on student learning. Faculty consider, adapt, and modify their courses, as appropriate, based on the feedback they received from their peers.|
Institutional Consultation and Reporting and Course (Re)Design occur concurrently with SAIL Coordinators reporting along with faculty, and faculty engaging in course redesign. A detailed description of each step is provided in subsequent sections of this Handbook.
SAIL is jointly coordinated through the Centre for Excellence in Learning and Teaching and Office of Quality Assurance at Thompson Rivers University, and co-led by an educational developer and quality assurance practitioner in partnership with faculty members from across the university.
SAIL Coordinators (Co-Principal Investigators): The primary role of SAIL Coordinators is to facilitate the components of the research project, including overseeing the Research Ethics Board (REB) process, facilitating the development of resources (i.e., institutional rubrics, rating sheets), providing Assessor Training, creating opportunities for team-building within the ILO Pods, maintaining the SAIL Moodle site, and promoting the dissemination of research findings. In addition, SAIL Coordinators respond to faculty and student inquiries related to the research project.
A SAIL Coordinator Check List from project start to close is available here: SAIL Coordinator Check List (PDF)
Faculty Members (Co-Investigators) organized as ILO Pods: Each ILO being assessed involves a community of practice of faculty members (“ILO Pod”) focused on student achievement of an ILO. A community of practice is formed when a group of people want to share common experiences and knowledge that are related to a particular area of expertise.
Faculty members teaching ILO designated courses volunteer to be part of SAIL. Within their ILO Pod, faculty members are co-researchers/co-assessors who:
- select one of their courses to be part of SAIL;
- collaboratively review (or co-create) an ILO rubric;
- select a course assignment;
- provide students with the consent process;
- assess a sampling of two colleagues’ assignments;
- review report of colleagues’ ratings; and,
- discuss recommendations resulting from the process during the Debrief.
The ILO Pods are fundamental to the SAIL methodology and were continued across both iterations of SAIL in 2020-21 and 2021-22. Each interdisciplinary ILO Pod has its own shared institutional rubric, series of meetings and opportunities for building trust, Moodle site, assessor training, and project debrief.
Administrative Support: Administrative support for scheduling meetings, file storage, monitoring the consent process, and responding to general inquiries is provided by the Office Coordinator within the Centre for Excellence in Learning and Teaching.
Students (Study Participants): Within the SAIL framework, students represent the study participants. Specifically, they are students enrolled in participating ILO-approved courses. Students contribute their data via course assignments through an REB-approved consent process. In addition, student perspectives were sought and integrated into the second iteration of SAIL as part of the Institutional Consultation and Reporting phase.
Koshy, E., Koshy, V., & Waterman, H. (2010). Action research in healthcare. Sage.
Levin, M., & Greenwood D. (2001). Pragmatic action research and the struggle to transform universities into learning communities. In P. Reason & H. Bradbury (Eds.) Handbook of action research: Participative inquiry and practice. SAGE.
Macintyre, C. (2000). The art of action research in the classroom. David Fulton Publishers.