Action Research Design prioritizing Qualitative Methods
SAIL methodology is based on an action research design, which has the dual benefit of generating practical solutions and empowering practitioners.
The process of action research involves engaging practitioners in systematic enquiry focused on generating solutions to practical problems and the subsequent development of activities to improve outcomes across multiple cycles (Koshy et al., 2010).
Quality action research, according to Levin and Greenwood (2001), is:
- contextually-grounded;
- addresses real life problems; and,
- involves participating practitioners constructing meaning through reflection that leads to action.
Each cycle of action research includes a plan-act-evaluate-reflect cycle. The cycle involves identifying the focus (e.g., a problem, changes for improvement, learning outcomes), developing a plan to assess progress, implementing the plan, reflecting on progress, and identifying necessary changes to improve and re-implement the plan thereby stimulating a new cycle (Koshy et al., 2010; Macintyre, 2000).
The SAIL Planning Cycle includes the application of qualitative methods (focus groups, rubric-based descriptive assessments, community consultations) with some initial quantitative descriptive analysis of consent rates. The eight-step SAIL Planning Cycle (Figure 1) is depicted below.
The first and second iterations of SAIL involved the use of faculty-developed institutional rubrics and the collection of data to evaluate the extent to which students were achieving Critical Thinking and Investigation, Social Responsibility, and Lifelong Learning in courses designated as meeting these three institutional learning outcomes (ILO) – also referred to as ILO-approved courses at the institution under study. The data was gathered through faculty applying a shared rubric to their peers’ course-embedded assignments.
Once the faculty-led communities of practice (“ILO Pods”) are formed for each of the ILOs being assessed the following steps are undertaken:
| Description | |
|---|---|
| 1. Preparation, Recruitment, and Launch |
Involves preparation, including consultation and approval, review of policies and compliance check (including Research Ethics Board), and recruitment of faculty members. |
| 2. Shared Rubrics | Involves the development (or refinement) of shared institutional rubrics. |
| 3. Student Consent and Artifacts | Involves the identification of appropriate student artifacts (course assignments) for assessment using the rubrics; and, notification to students of the consent process (e.g., opt-in or opt-out). |
| 4. Assessor Training | Faculty participate in Assessor Training delivered by an educational developer and quality assurance practitioner. |
| 5. Assessment of Student Learning | Faculty assess two of their peers’ assignments using the shared rubric and rating sheet. |
| 6. Debrief | Faculty participate in a focus group and are guided through semi-structured prompts led by the SAIL Coordinators to determine the efficacy of the SAIL pilot project. |
| 7. Institutional Consultation and Reporting | Findings and recommendations are drafted, reviewed, and disseminated. |
| 8. Course (Re)Design Implications | Faculty review and discuss their peers’ assessment and feedback of the evidence provided on student learning. Faculty consider, adapt, and modify their courses, as appropriate, based on the feedback they received from their peers. |
Institutional Consultation and Reporting and Course (Re)Design occur concurrently with SAIL Coordinators reporting along with faculty, and faculty engaging in course redesign. A detailed description of each step is provided in subsequent sections of this Handbook.
Coordination
SAIL was originally co-led by an educational developer and quality assurance practitioner; however, in its third iteration was driven by teaching faculty in partnership with faculty researchers from across the university.
Members of SAIL Pilot Projects
SAIL Coordinators (Co-Principal Investigators): The primary role of SAIL Coordinators is to facilitate the components of the research project, including overseeing the Research Ethics Board (REB) process, facilitating the development of resources (i.e., institutional rubrics, rating sheets), providing Assessor Training, creating opportunities for team-building within the ILO Pods, maintaining the SAIL shared site (e.g., Moodle, Teams, etc.), and promoting the dissemination of research findings. In addition, SAIL Coordinators respond to faculty and student inquiries related to the research project.
A SAIL Coordinator Check List from project start to close is available here: SAIL Coordinator Check List (PDF)
Faculty Members (Co-Investigators) organized as ILO Pods: Each ILO being assessed involves a community of practice of faculty members (“ILO Pod”) focused on student achievement of an ILO. A community of practice is formed when a group of people want to share common experiences and knowledge that are related to a particular area of expertise.
Faculty members teaching ILO designated courses volunteer to be co-investigators collaborating on SAIL as part of their research and scholarship. Within their ILO Pod, faculty members are co-researchers/co-assessors who:
- select one of their courses to be part of SAIL;
- collaboratively review (or co-create) an ILO rubric;
- select a course assignment;
- provide students with the consent process;
- assess a sampling of two colleagues’ assignments;
- review report of colleagues’ ratings; and,
- discuss recommendations resulting from the process during the Debrief.
The ILO Pods are fundamental to the SAIL methodology and were continued across all iterations of the action research approach. Each interdisciplinary ILO Pod has its own shared institutional rubric, series of meetings and opportunities for building trust, shared website, assessor training, and project debrief/collaborative self-study process.
Students (Study Participants): Within the SAIL framework, students represent the study participants. Specifically, they are students enrolled in participating ILO-approved courses. Students contribute their data via course assignments through an REB-approved consent process (REB No. 102,637). In addition, student perspectives were sought and integrated into the second iteration of SAIL as part of the Institutional Consultation and Reporting phase.
References
Koshy, E., Koshy, V., & Waterman, H. (2010). Action research in healthcare. Sage.
Levin, M., & Greenwood D. (2001). Pragmatic action research and the struggle to transform universities into learning communities. In P. Reason & H. Bradbury (Eds.) Handbook of action research: Participative inquiry and practice. SAGE.
Macintyre, C. (2000). The art of action research in the classroom. David Fulton Publishers.
