study design
The realist approach attempts to answer the question “what works, why and how, under what circumstances”, using realism as a paradigm that lies between positivism and social constructivism [20, 21]. In this approach, we first set up a working hypothesis that we want to examine in the context of an educational program. Next, we examine, test and refine which mechanisms work under which conditions (context) and with which interventions (including opportunities or resources) using purposefully collected quantitative or qualitative data, from which we can describe explainable outcomes in an iterative manner. This is described by the formula context + mechanism= Result(based on previous literature, the components are referred to as CMOs, an acronym for Context Mechanism Outcome) [22]. The key to validating CMOs lies in the consistency and integration of the CMOs refined from the working hypotheses on the one hand with the data collected on the basis of these hypotheses on the other hand [23]. This analysis process is detailed in the Realistic Approach section below.
Curriculum of medical students in Japan
We briefly describe the curriculum for medical education in Japan here to facilitate understanding of the learning context [24]. In Japan, students enter medical school after high school. The curriculum is presented in a six-year program beginning in April. In the first and second years, students begin with lectures and practical exercises in basic medical education such as anatomy and physiology. In the third and fourth years, they study clinical medicine, including internal medicine and surgery. Students must pass a CBT (Computer-Based Testing) and OSCE (Objective Structured Clinical Examination) prior to clinical practice in order to receive the title of Student Physician and be accepted for clinical practice. Clinical practice at the university in this degree is conducted from January of the fourth year to November of the sixth year for a total of 72 weeks. JH and colleagues at other universities have previously developed a community-based clinical practice medical education (CBME) curriculum in which 16 to 19 students rotate every four weeks, spending one to two weeks at a time in clinics or small hospitals to practice SDH in to learn from local communities [25]. This type of program is generally offered as part of the general medicine component of clinical practice in the fourth or fifth year at some Japanese universities. In the present study, we conducted a program evaluation focused on online community diagnosis in student-friendly hometowns at X University that differs from the previous literature [25].
General medicine in clinical practice
The component General Practice in Clinical Practice at University X was previously conducted from January of the fourth year to December of the fifth year. Clinical rotations consisted of 5-6 students every 2 weeks. Due to the corona virus epidemic, we switched to a full online rotation operation from February to May 2020. Although clinical rotation operations at the University Hospital resumed in August 2020, some online rotations remained. We have therefore developed a community diagnostic program to educate medical students about health issues in the community through online general practice training. The delivery of this online component and this study began in August 2020. JH is a general practitioner with a PhD and has published several qualitative studies, including studies using real-world approaches; TA is a General Practitioner with 6 years experience in community hospitals and clinics including qualitative research through reports; SF has been a doctor for 40 years, holds a doctorate and is a program director in a department of general medicine; and JH is involved in training in collaboration with a general medicine department and has various ways of communicating with the other authors.
learning theory
Community-based medical education has used experiential learning, communities of practice, and situated learning as theoretical frameworks [25]. However, few learning theories clearly explain the reality of the learning experience of medical students learning about their own community [16]. Therefore, using variation theory, we considered core learning as distinction and variation in situations with temporal, spatial, and social dimensions [26, 27]. We have defined learning about a community as “the important identification of things that occur in the interaction between the learner and the learning object; and the variations that arise in this interaction are what bring to mind the essential properties of things based on the theory of variations.” This helped clarify the learning experience of medical students.
learning goals
Two learning objectives were set.
Understand the purpose and methods of community diagnostics.
Identify the characteristics of local communities and analyze community health issues specific to the local community.
programming
In a one-hour zoom-based lecture on the first day of clinical practice, the importance of community diagnosis was presented along with concrete examples. The students could select their own place of residence or an area they are familiar with, such as their hometown, for the group diagnosis. After a week, the medical students presented their researched joint diagnosis and received feedback from colleagues and an author (JH). On the final day of the two-week program, each student presented a structural community diagnosis report that included their own feedback-based assumptions or opinions and submitted a revised report. To avoid researcher (JH) bias, TA surveyed the medical students on the last day of the program to get their feedback on their learning instead of JH.
student assessment
The summative assessment was carried out by the first author using a rubric. This assessment included evidence from the interim and final presentations as well as structured reports from the community diagnosis. This rubric was shared with the medical students on the first day. The author participated in observing the students’ presentations on the intermediate and final days of the program; their listening attitude during their colleagues’ presentations; and how they responded to feedback from peers and the author. Formative assessment was also conducted during the intermediate and final presentations. Feedback was mainly given on the validity of what was examined in the community diagnosis, the logic of its explanation of community problems, and the relevance and feasibility of the action plan.
attitude and study participants
The site of this study was the Department of General Medicine at X University in Tokyo, Japan, from August 2020 to December 2021. The academic year of Japanese universities starts in April. Our university begins clinical practice in January of Year 4 and the subsequent 24 months through December of Year 5 is basic clinical practice. The proportion of female medical students in Japan is said to be low; In 2018, only 21.9% of doctors were women [28]. Study participants included 4th and 5th year medical students who participated in a clinical internship in general practice in the basic clinical practice program.
Realistic Approach
The analytical procedure was performed according to Pawson’s four steps [22]. First, we formulated a working hypothesis that we wanted to examine in the program. Working hypotheses on the program were formulated using deductive and inductive methods [22]. On this basis, using purposefully collected, mainly qualitative data, we examined, tested and refined which mechanisms worked under which conditions (context), with which interventions (including possibilities or resources) and from which explainable results could be described and verified iteratively . This heuristic is meant to remind us to think of realistic evaluations in terms of constructs and not in terms of formulas like in mathematics.
Creation of a working hypothesis about the components of CMOs
Context is the condition under which the program is introduced and refers to the mechanism that takes into account “under what circumstances” the program will work. Mechanism is the process by which an individual interprets and responds to an intervention. Outcomes are how a set of outcomes leads to an effect or change. We started with hypotheses about possible mechanisms. Astbury and Leeuw reported that a clear distinction should be made between mechanisms and program interventions [29]. For example, an outcome might be an increase in learner knowledge or readiness resulting from a mechanism resulting from an educational intervention. In other words, the mechanism is an explanatory model that can be described in terms of multiple contextual and educational intervention variables. Dalkin et al. used the method described by Pawson et al. featured formula,context+mechanism=Result [22]. Regarding the mechanism of program intervention and the mechanism as program-related inference, Dalkin et al. proposed a method of describing thecontext+Mechanism (Intervention) → Mechanism (Inference)=Result [30]. With this representation, mechanisms are explained both by concrete facts and by interpretive conclusions. The contexts in which program interventions are introduced vary, as do the patterns of mechanisms (inferences) and activated outcomes that emerge. As such, CMOs describe multiple patterns of how the various components of a program harmonize and integrate. To describe these patterns, we used here the observational assessment and structural community diagnosis reports submitted by the students on the last day of the program.
observation and verification
The working hypothesis was tested by collecting data on GMOs. At the end of the financial year, a program evaluation was carried out on the basis of the oral and written reflections and reports from the medical students. The authors confirmed whether a number of CMOs were able to comprehensively explain the learning patterns of the community diagnostic program as an assessment. In testing for consistency and integration, we did not just focus on a single outcome but examined whether CMOs as learning patterns of community diagnosis under different combinations of learning outcomes with contexts and mechanisms allowed readers to transfer the findings to other settings.
Clarification of the CMOs
This validation and refinement process resulted in a pattern of CMOs, namely a set of mechanisms and learning outcomes that could explain the complex learning process based on multiple contexts. Ongoing validation and refinement of the CMO patterns was performed using data collected over the two years [31].
Ethical Approval and Consent to Participate
This study was approved by the Ethics Committee of Keio University (approval number: 20211157) and conducted in accordance with the Declaration of Helsinki. All participants had the opportunity to unsubscribe on the Keio University Medical Training Center website. Informed consent was obtained from all participants.
