Review Article
Problem-Based Learning in Dental Education: A Systematic Review of the Literature Seyed Hossein Bassir, D.D.S.; Pooyan Sadr-Eshkevari, D.D.S.; Shaden Amirikhorheh, D.D.S.; Nadeem Y. Karimbux, D.M.D., M.M.Sc. Abstract: The purpose of this systematic review was to compare the effectiveness of problem-based learning (PBL) with that of traditional (non-PBL) approaches in dental education. The search strategy included electronic and manual searches of studies published up to October 2012. The PICO (Population, Intervention, Comparator, and Outcome) framework was utilized to guide the inclusion or exclusion of studies. The search strategy identified 436 articles, seventeen of which met the inclusion criteria. No randomized controlled trial was found comparing the effectiveness of PBL with that of lecture-based approach at the level of an entire curriculum. Three randomized controlled trials had evaluated the effectiveness of PBL at a single course level. The quality assessment rated four studies as being of moderate quality, while the other studies were assessed as being of weak quality. This review concludes that there are a very limited number of well-designed controlled studies evaluating the effectiveness of PBL in dental education. The data in those studies reveal that PBL does not negatively influence the acquisition of factual knowledge in dental students and PBL enhances the ability of students in applying their knowledge to clinical situations. In addition, PBL positively affects students’ perceived preparedness. Dr. Bassir is a D.M.Sc. candidate, Division of Periodontology, Department of Oral Medicine, Infection, and Immunity, Harvard School of Dental Medicine; Dr. Sadr-Eshkevari is in the Professional Program for International Dentists, School of Dentistry, University of California, Los Angeles; Dr. Amirikhorheh is Researcher, Dental Research Center, School of Dentistry, Guilan University of Medical Sciences, Rasht, Iran; and Dr. Karimbux is Professor and Associate Dean for Academic Affairs, Tufts University School of Dental Medicine, Boston, MA. Direct correspondence and requests for reprints to Dr. Seyed Hossein Bassir, Division of Periodontology, Harvard School of Dental Medicine, 188 Longwood Ave., Boston, MA 02115;
[email protected]. Keywords: comparative effectiveness research, dental curriculum, dental education, evidence-based dentistry, educational modules, problem-based learning, professional education Submitted for publication 2/22/13; accepted 4/14/13
P
roblem-based learning (PBL) is defined as an approach in which a problem serves as the stimulus for active learning. The PBL approach is student-centered in the way that students define the problem and establish learning objectives required to develop their understanding of the problem. This pedagogy is based on small groups of students working together and collaborating with faculty facilitators to achieve understanding.1,2 The main objectives of PBL have therefore been to enhance adult learning skills by engaging students through self-direction and problem-solving and also to nurture clinical reasoning, teamwork, and communication skills. These skill sets promote lifelong learning and better preparation of students for their professional careers.3-6 As an alternative to the traditional pedagogical model, PBL was introduced into health sciences edu-
98
cation at McMaster University in 1969.1,7 It has since been implemented in many medical school curricula and other professional training programs around the world. PBL was first introduced to dental education at the Faculty of Odontology in Malmö, Sweden, in 1990.8 Implementation of PBL in dental education gradually increased during the 1990s and 2000s,9 so that in 2009 almost half of U.S. and Canadian dental schools had adopted this pedagogical method in whole or in part.10 PBL has been implemented in various types and at various levels. Some universities such as Maastricht adopted a pure PBL curriculum that involves no lectures at all; some like McMaster University developed a PBL curriculum with few lectures, whereas some others like Harvard Medical School implemented a hybrid PBL curriculum that provides more structured didactic learning supported
Journal of Dental Education ■ Volume 78, Number 1
by PBL sessions. In some schools, there is a PBL approach running at the whole curriculum level, while in others PBL is presented at a single course level within a lecture-based curriculum. A large number of studies have evaluated the outcomes of these PBL interventions in medical curricula.6,11-15 However, these studies have demonstrated contradictory results regarding the effectiveness of PBL. Although medical education and dental education have conceptual similarities, it should be emphasized that the skills medical and dental students need to acquire are not always the same. For instance, clinical skills in medical education are usually defined as clinical reasoning and clinical problem-solving that are mostly related to conducting an appropriate physical examination and establishing a proper diagnosis.16 Even though those clinical skills are also crucial for dentists, clinical skills in general dentistry involve mainly mechanical hand activities that rely on developing psychomotor skills.17,18 The differences in required skills between dental and medical students make the focus of dental education quite different from that of medical education. Accordingly, the feasibility of applying the findings of medical education studies, including those evaluating the effectiveness of PBL, to dental education may be limited. Although some systematic reviews in the medical field have compared the effectiveness of PBL with that of traditional pedagogy, there is still a need for a systematic review of these studies in the dental field. To the best of our knowledge, no systematic reviews of the literature have been conducted that compare the PBL pedagogy with the traditional method in dental education. The aim of this study was therefore to perform a systematic review of studies to compare the effectiveness of PBL with that of traditional (nonPBL) approaches in dental education.
Materials and Methods We defined our research question as follows: Are there differences in outcomes when comparing traditional (non-PBL) curricula to curricula that have PBL as a component? To identify relevant studies, the following electronic databases were searched from the start of the database to October 2012: MEDLINE, Educational Resources Information Center (ERIC), PsychInfo, CINAHL, and ISI Web of Science. The following combinations of keywords were used: “Problem-based learning” AND “Dental Education”; “Problem-based learning” AND “Dentistry.”
January 2014 ■ Journal of Dental Education
Both keywords and MeSH terms were employed in the Medline search. To supplement the searches, the tables of contents of two key dental education journals (Journal of Dental Education and European Journal of Dental Education) were searched for relevant articles. In addition, a manual search of relevant references in all included studies was performed. The PICO (Population, Intervention, Comparator, and Outcome) framework was utilized to guide the inclusion or exclusion of studies in the review. The study population was limited to predoctoral and postdoctoral dental students. The PBL group included any schools that use any method of PBL either as a curriculum-wide intervention or as a single course intervention. In terms of the definition of PBL, the study was included if the authors described their intervention as PBL. The studies had to have a control group consisting of traditional (non-PBL) educational approaches. Included studies had to provide quantitative outcome measures of PBL effectiveness. In order to present all existing evidence on the effectiveness of PBL, all study designs with a control or comparison were considered for inclusion in our review. Studies were excluded if they were not available in English, did not compare PBL with traditional educational method, did not clearly describe the experimental methodology or outcome parameters, or did not provide statistical analysis. Two reviewers independently screened all articles identified by the electronic searches. Potential articles were discarded at the title level if they were clearly irrelevant to the study. Abstracts of potentially relevant articles were screened for relevance of the reported intervention and study design. Remaining articles were then read in full to decide whether they met all inclusion criteria. Disagreements regarding the inclusion of the studies were resolved through discussion and consensus. The quality of the included studies was assessed by means of the Quality Assessment Tool for Quantitative Studies, developed for the Effective Public Health Practice Project (EPHPP), Canada, as adapted by Thomas et al.19 The EPHPP tool has been judged suitable to be used in systematic reviews of effectiveness,20 and it can be used to assess a variety of intervention designs including randomized controlled trials, quasi-experimental studies, and uncontrolled studies. It has been found to have content and construct validity and excellent interrater reliability.19,21,22 This tool consists of six criteria: selection bias, study design, confounders, blinding, data collection method, and withdrawals/dropouts. Each criterion was
99
rated as strong, moderate, or weak according to the dictionary of the tool; the overall assessment of the study is determined by assessing these ratings. According to the guidelines for the tool, studies with no weak rating and four strong ratings are classified as “strong”; studies with fewer than four strong ratings and one weak rating are classified as “moderate”; and studies with two or more weak ratings are classified as “weak.” Two reviewers independently performed the assessment of the quality of the included studies. Any discrepancies in quality ratings were resolved by discussion and consensus. Two reviewers did data extraction independently for the included studies, and any discrepancies were resolved by discussion and consensus. The following data were extracted from each included study: first author, journal, publication year, setting and country of the study, study design, type of intervention, study population, sample size, evaluation tool, outcome variables, and results. A meta-analysis was not appropriate due to the heterogeneity across the included studies in the design, type of intervention, study population, and outcome variables.
Results A flow diagram of the search results is shown in Figure 1. The initial search of the electronic databases identified 566 articles of which 131 were duplicates. The manual search yielded only one new study. One hundred twenty-five citations were excluded on the basis of the title because they were reviews, editorials, or letters or citations were irrelevant to PBL in dental education. Screening of the abstracts of the articles reduced the number of studies to forty-two. The full-text versions of all remaining articles were obtained and carefully reviewed. Twenty-five articles on further reading did not meet the inclusion criteria because of the lack of comparison PBL with traditional educational approach,23-42 lack of a PBL intervention,43,44 lack of quantitative outcome measures,45 failure to provide statistical analysis,46 or reporting of overlapping data with another article.47 Therefore, seventeen articles were included in the systematic review.18,48-63 Table 1 shows the characteristics of the included studies. Four studies evaluated the effectiveness of PBL at a single course level.48,52,56,62 Three of these studies used a randomized controlled trial design,52,56,62 while the other study did not randomize the participants.48 The results of quality assessment classified all of
100
these studies as being of weak quality. Table 2 shows the evaluation tools, outcome variables, results, and quality assessment of these four studies. There were no randomized controlled trials that compared the effectiveness of PBL with that of lecture-based approach at a whole curriculum level. Thirteen studies were found that compared the educational approaches.18,49-51,53-55,57-61,63 These studies used concurrent cohorts at different schools,50,53,54,59 used non-concurrent cohorts at the same school,49,51,58,63 or employed a retrospective design.18,55,57,60,61 The quality assessment rated four of these studies as being of moderate quality,55,57,60,61 while the other studies were assessed as being of weak quality. Table 3 shows the evaluation tools, outcome variables, results, and quality assessment of the studies evaluating the effectiveness of PBL at whole curriculum level.
Discussion This systematic review of studies in the field of dental education assessed whether there are differences in quantitative outcomes when comparing traditional (non-PBL) curricula to curricula with a PBL component. The results of this review demonstrated that there is no randomized clinical trial that compared these educational methodologies at the whole curriculum level. At the single course level, three randomized clinical trials compared traditional and PBL approaches, and these results were mostly in favor of PBL intervention.52,56,62 The other included studies employed non-randomized experimental, quasi-experimental, or retrospective designs and provided inconsistent findings regarding the effectiveness of PBL. The heterogeneity across these studies in outcome variables, study design, level and type of PBL intervention, study population, and control group could be responsible for the inconsistent results. The outcome variables that have been used to assess the effectiveness of PBL intervention in dental education could be categorized under such headings as variables assessing knowledge acquisition, knowledge application, clinical hand skills, perceived preparedness, and a wide range of behavioral characteristics of students and graduates. In the medical education literature, some educators have questioned the performance of PBL students on standardized tests of factual knowledge and have argued that PBL has a negative effect on the acquisition of factual knowledge.11,12,64 On the other
Journal of Dental Education ■ Volume 78, Number 1
Medline (n=412)
Web of Science (n=62)
CINAHL (n=37)
Records a er duplicates eliminated (n=436)
ERIC (n=35)
PsycINFO (n=20)
Manual search (n=1)
Excluded a er tle screening (n=125)
Abstracts screened (n=311)
Excluded a er abstract screening (n=269)
Full text screened (n=42)
Studies included in the systema�c review (n=17)
Excluded a er full text screening (n=25) • not compare PBL with tradi�onal curriculum (n=20) • not PBL Interven�on (n=2) • not quan�ta�ve study (n=1) • no sta�s�cal analysis (n=1) • publica�ons with overlapping data (n=1)
Figure 1. Study selection flow diagram of the systematic review
January 2014 ■ Journal of Dental Education
101
102
Journal of Dental Education ■ Volume 78, Number 1
Eur J Dent Educ
Greenwood et al.
J Dent Educ
Susarla et al.
Eur J Dent Educ
J Dent Educ
Rich et al.
Last et al.
Adv Health Sci Educ Theory Pract
J Dent Educ
Thammasitboon et al.
Reich et al.
J Dent Educ
Polychronopoulou et al.
J Dent Educ
Med Oral Patol Oral Cir Bucal
Moreno-López et al.
Sukotjo et al.
Eur J Dent Educ
J Dent Educ
Callis et al.
Polyzois et al.
J Dent Educ
J Oral Maxillofac Surg
Zhang et al.
Yiu et al.
Journal
First Author
1999
2001
2003
2005
2007
2007
2007
2009
2009
2010
2010
2011
2012
Year
Table 1. Descriptive data on included studies
University of Toronto Faculty of Dentistry; University of Adelaide Department of Dentistry
University of Liverpool School of Dentistry
Harvard School of Dental Medicine
University of Southern California School of Dentistry
NR
Harvard School of Dental Medicine
Harvard School of Dental Medicine (HSDM); non-HSDM
Six European dental schools
Universidad Europea of Madrid
Dublin Dental School and Hospital
Indiana University School of Dentistry (IUSD); Baylor College of Dentistry (BCD)
University of Hong Kong Faculty of Dentistry
Shanghai Jiao Tong University School of Stomatology
Setting
Canada Australia
UK
USA
USA
NR
USA
USA
Croatia, Greece, Ireland, Slovenia, Spain, Sweden
Spain
Ireland
USA
China
China
Country a
Dental students: PBL: Classes of 2005 and 2006 LBS: Classes of 2003 and 2004
Dental students from a same year
Dental students: h-PBL: Classes of 2005 and 2006 LBS: Classes of 1980-91
Dental residents across U.S.: h-PBL: HSDM graduates between 2002-4 participated in residency programs LBS: their co-residents
Dental students in the six schools: PBL: two schools in Ireland and Sweden LBS: the other four schools
5th-yr dental students
Dental graduates: PBL: Classes of 2002-5 h-PBL: Classes of 1998-2001 LBS: Classes of 1994-7
3rd-yr dental students: h-PBL: IUSD 2007-8 and 2008-9 classes LBS: BCD 2007-8 class
Dental graduates: PBL: Classes of 2004-8 LBS: Classes of 1997-2001
4th-yr dental students
Population
WC: PBL vs. LBS
WC: PBL vs. LBS
Dental students at graduation: PBL: Adelaide Class of 1997 LBS: Toronto Classes of 1996 and 1997
2nd-yr dental students: PBL: students tested in 1999 and 2000 LBS: students tested in 1997 and 1998
WC: h-PBL vs. h-PBL Dental students: only in basic sciences h-PBL: Classes of 1998b-2002 vs. LBS h-PBL in basic sciences: Classes of 1992-8a LBS: Classes of 1980-91
WC: PBL vs. LBS
SC: h-PBL vs. LBS
WC: h-PBL vs. LBS
WC: h-PBL vs. LBS
WC: PBL vs. LBS
SC: PBL vs. LBS
WC: PBL vs. h-PBL vs. LBS
WC: h-PBL vs. LBS
WC: PBL vs. LBS
SC: h-PBL vs. LBS
Intervention Type
38 vs. 93
109 vs. 102
NR
274 vs. 234; 134 vs. 233b
47 vs. 50
70 vs. NR
42 vs. 38
299 vs. 1193
15 vs. 36
76 vs. 76 vs. 76
31 vs. 40
159 vs. 104
43 vs. 44
Sample Size
Students entered study as 4th year, and last assessment was at end of their 5th year. Scores of PBL class of 2006 were not included in statistical analysis for one of outcome variables (clinical competency examination). Class of 1992 was excluded from study since it represented a transition class (incomplete PBL). d Students entered study as 2nd year, and last assessment was at end of their 3rd year. c
b
a
PBL=problem-based learning; h-PBL= hybrid problem-based learning; LBS= lecture-based system; NR=not reported; SC=single course; WC=whole curriculum
1st-yr: 46 vs. 39; 2nd-yr: 23 vs. NR; 3rd-yr: 25 vs. NR 1st-, 2nd-, and 3rd-yr dental students: PBL: students tested in 1993-5 LBS: students tested in 1922 WC: PBL vs. LBS Australia University of Adelaide Department of Dentistry J Dent Educ Townsend et al.
1997
20 vs. 20 2nd-yr dental studentsd SC: h-PBL vs. LBS Canada University of British Columbia Faculty of Dentistry J Dent Educ Walton et al.
1997
42 vs. 20 3rd-yr dental students: h-PBL: Classes of 1993 and 1994 LBS: Class of 1991c WC: h-PBL vs. LBS USA Harvard School of Dental Medicine J Dent Educ Login et al.
1997
12 vs. 130 Dental students matriculated in 1995 WC: PBL vs. LBS USA University of Southern California School of Dentistry 1998 J Dent Educ Shuler et al.
January 2014 ■ Journal of Dental Education
hand, some authors have claimed that standardized tests like multiple-choice exams are not appropriate for assessment of the deep levels of understanding expected to be emphasized in PBL.65,66 However, in the dental field, since the acquisition of factual knowledge of basic sciences is still thought to be a fundamental requirement for students, several studies have investigated the effectiveness of PBL through multiple-choice examinations.48,55,57,58,60 These studies demonstrated that PBL students not only performed as well as traditional students in the multiple-choice exams on factual knowledge,48,58 but also that the PBL students scored significantly higher than traditional students when a standard examination such as the National Board Dental Examination was used as a way of evaluating the effectiveness of PBL.55,57,60 In addition, PBL students demonstrated significantly superior performance when the application of knowledge was evaluated through objective structured clinical examinations,18 clinical skill tests,48,52 and comprehensive examinations on the integration of basic sciences and clinical sciences.50 These data suggest that PBL could increase the abilities of dental students to acquire factual knowledge and to apply the knowledge to clinical situations. Clinical hand skills are considered of crucial importance for dental students. According to the results of our study, there is no evidence suggesting PBL has positive effects on the development of clinical hand skills of dental students. Only two studies evaluated the effect of PBL on clinical hands skill development (one on preparing CAD/CAM ceramic inlays56 and the other on performing nonsurgical periodontal treatment18). These studies failed to reveal any statistically significant differences between the PBL and traditional students. This currently available limited data might suggest that the implementation of PBL has not had a detrimental effect on the clinical hand skills development of dental students. Another outcome variable that has been used to assess the effectiveness of PBL is the self-perceived preparedness of dental students or graduates.49,54,59,62 Most of the studies to date showed that PBL students felt better prepared than non-PBL students.54,59,62 Although the results of the self-assessment measurement might not necessarily reflect the true performance, these data show that PBL has a favorable effect on enhancing students’ confidence in their abilities to practice dentistry, which can positively affect their future career. Although it is generally accepted that randomized controlled trials (RCTs) provide the highest
103
104
Journal of Dental Education ■ Volume 78, Number 1
Zhang et al., 2012
Walton et al., 1997
1. Patient management ability after 1-yr clinical rotation 2. Knowledge on subjects before and after clinical rotationb 3. Subjective feelings about learning experience in a) study interest, b) helpfulness of course, c) improvement in knowledge and clinical skill, d) motivation for cooperating, e) confidence in practice, f) spent time in library
2. Multiple-choice examinations
3. Questionnaire using VAS
2. Faculty evaluation on same items
2. SAA
1. Clinical skill test
1. Students perceived preparedness and progress in a) understanding principles of treatment planning, b) making accurate diagnostic records, c) understanding difference of treatment options and treatment plans, d) improvement of treatment planning skills, e) improvement in producing accurate recordsa
2. Course evaluation
1. 5-item statement assessment using VAS
1. Quality of MOD inlays
2. 25-item seven-point Likert scale questionnaire
3. Time devoted by teachers to a) classes, b) study group, c) tutorship, d) teacherhours:student ratio
3. SAA
1. Faculty evaluation
2. Time devoted by students to a) tutorials, b) discussion, c) study, d) information retrieval
2. Questionnaire
Outcome Variables 1. Students’ examination marks for subject
Evaluation Tool
1. Faculty evaluation
Results
3. Higher in PBL for (a)**, (b)*, (d)*, (e)**, (f)**; NSD for (c)
2..NSD before clinical rotation (after intervention); higher scores in LBS after clinical rotation*
1. Higher scores in PBL**
2. Higher in PBL for (b) and (d)***; higher in LBS for (c)***; NSD in other items
1. Higher in PBL for (a), (b), (e)***; NSD in other items
2. NSD in overall course assessment; PBL was rated higher in possibility of asking questions*, equipment conditions*, learning atmosphere*, student/teacher relationship*, help during course*, and use of multimedia*. LBS was rated higher in personal success*.
1. NSD
3. NSD
2. PBL students devote more time to (a)**, (b)**, (d)*, and less time to (c)*
1. Higher in PBL*
Weak
Weak
Weak
Weak
Study Quality
b
a
Intervention was carried out in 2nd-yr, but assessment was carried out at end of 3rd-yr. Intervention was carried out in 4th-yr, but evaluation was carried out at beginning and end of 5th-yr. *Significant difference (p<0.05) **Significant difference (p<0.01) ***Considered significant by Walton et al. (p<0.1)
CT=controlled trial (non-randomized); LBS=lecture-based system; NSD=no significant difference; PBL=problem-based learning; RCT=randomized controlled trial; SAA=same as above; VAS=visual analog scales
CT
RCT
Reich et al., 2007
Author, Date
Moreno-López et al., 2009
Study Design
Table 2. Summary of outcome variables, results, and quality assessment of included studies (single course intervention)
level of evidence for the effectiveness of an intervention, it is not always possible to use an RCT approach for educational research. In our review, only three RCTs were found assessing the effectiveness of PBL.52,56,62 These studies are also at a high risk of bias since the results of quality assessment rated them as being of weak quality. Our results revealed that the majority of the included studies used a quasi-experimental design in which the control groups were either historical controls from the same school before the curriculum change49,51,58,63 or cohort controls from another school with a non-PBL curriculum.50,53,54,59 These quasi-experimental studies are subject to a higher risk of bias than RCT studies. This is attributable to the baseline differences or other confounding variables such as differences in admission criteria between the schools or differences within the same school before and after PBL implementation.67 Thus, it is important to emphasize that the results of studies evaluating the effectiveness of PBL cannot be interpreted without consideration of the design of the studies. Another factor that should be taken into account when interpreting the literature on the effectiveness of PBL is the level of implementation of this intervention. Our review found that four studies evaluated the effectiveness of PBL when it was implemented as a single course intervention in a lecture-based curriculum.48,52,56,62 Most of these studies reported results that favored the PBL intervention.48,52,62 This finding is consistent with that of Polyzois et al., who systematically reviewed studies investigating the effectiveness of PBL in predoctoral health education and found that the results of a single PBL intervention in a traditional curriculum were almost in support of PBL intervention.15 Appraising the efficacy of PBL at a single course level could offer advantages such as a more controlled environment, which might limit the confounding factors presented at a whole curriculum level. However, it has been argued that the aims of PBL can truly be achieved only when it is employed as a curriculum-wide pedagogy.12,66 The studies that have made comparisons between PBL and traditional curricula at a whole curriculum level present inconsistent results regarding the effectiveness of PBL, and they suffer from methodological limitations because of their study design. Although investigating curriculum-wide interventions is difficult due to the complex and multifactorial nature of dental curricula, better-designed curriculum-wide studies are required to draw a clear conclusion regarding the effectiveness of PBL in dental education.
January 2014 ■ Journal of Dental Education
The heterogeneity of the type and definition of the implemented PBL among the included studies is another biasing factor. The type of implementation of PBL varies from a pure PBL curriculum to a hybrid one. This lack of uniformity could be responsible for the inconsistency between the results of the studies. Furthermore, the broad definition and therefore the inconsistent interpretation of PBL pose a challenge in all PBL studies.1,14 Most of the included studies failed to provide a detailed description of design or delivery of PBL. Hence, it is not possible to determine if there is a consistent intervention in all the studies. Our review had to accept the authors’ claims about having implemented PBL into their studies; this could be considered a possible limitation of our study. In addition to issues related to the concept of the PBL approach, there are some practical issues that might affect the results of PBL studies. Some of these issues concern the quality of PBL “cases” or “problems,” the number of students in each study group, the experience of “facilitator” or “tutors,” and the suitability and availability of physical resources. Accordingly, as noted by Winning and Townsend, there can be “good” or “bad” implementation, practice, or presentation of PBL.66 It should therefore be emphasized that the results of PBL studies are likely to be influenced by the performance of the programs and might not necessarily reflect the inherent effectiveness of the educational approach itself.
Conclusion This study found that the number of welldesigned controlled studies in dental education on the effectiveness of PBL, especially at a whole curriculum level, is very limited, which affects the level of evidence. In addition, the majority of the studies were rated as being of weak quality. Despite this, the evidence from this analysis suggests that PBL does not adversely affect the acquisition of factual knowledge in dental students. It seems that PBL has a positive effect on students’ perceived preparedness and their confidence to practice dentistry; this pedagogy could also improve the ability of students in applying their knowledge in the clinical setting. To date, there is no convincing evidence in support of PBL for developing the clinical hand skills of dental students. While the limitations inherent to conducting research in the field of education should not be overlooked, we believe that further well-designed studies on this topic are needed.
105
106
63-item four-point Likert scale questionnaire
Four-point rating scale postal questionnaire
Postal questionnaire with 34 questions in four categories: simple questions, multiplechoice questions, VASs, or combinations.
Greenwood et al., 1999
Yiu et al., 2011
Polyzois et al., 2010
2. NSD in 8 domains; PBL graduates felt less well prepared in orthodontics**. NSD in 54 competencies; PBL graduates felt less prepared in 5 competencies: “replacing teeth with complete dentures”**, “reestablishing an occlusal vertical dimension”**, “performing minor tooth movement”**, “recognizing, reporting, and following up neglect and abuse cases”**, “extracting impacted third molars”**.
2. Self-perceived preparedness of graduates in 9 domains of dental practice including 59 competencies
1. Questions on professional activities, postgraduate 1. NSD training, specialty certification, practice type, hospital affiliation, computer use, time spent in CDEs, participation in CDE courses, local conferences, meetings, etc.
Long-term attitude towards lifelong learning by:
1. PBL graduates had less frequently changed practice type**; NSD on other items.
NSD for 40 competencies; PBL students feel more confident for: partial dentures*, complete dentures**, endo multi-rooted*, evaluation of dental literature**, child management*, communication with staff** and colleagues**, oral pathologies**, chronic orofacial pain*, ex impacted 3rd molar**, resin bonded bridges**. LBS students feel more confident for: performing deep scaling and root planing**, managing anxious patients**, treating early perio diseases**, minor tooth movements*, fixed bridges**, discuss payment**, periosurgery**, full arch alignment**.
PBL graduates rated themselves more highly than non-PBL graduates on (a)**, (b)**, (c)*, (e)**, (f)**, (g)**; NSD on other items.
PBL students felt less stress for (a), (c), (e) compared to LBS students.
PBL students performed better on case I*; NSD on case II.
Results
1. Practice characteristics including type of dental practice, practice change, and practice nature (solo vs. non-solo)
Self-perceived preparedness at graduation on 59 competencies
Self-assessment of performance in the program for: a) independent learning, b) communication with patients, c) teamwork, d) communication with staff, e) performance in small group, f) critical thinking, g) selfassessment, h) patient education, i) specialty specific knowledge, j) clinical skills, k) general dental knowledge, l) preclinical skills
Four-point scale questionnaire on 12 competencies
Thammasitboon et al., 2007
Essay examination by two Integrating basic science knowledge with two clinical clinical cases and open-ended cases: questions Case I: physiology, neurology, pharmacology, biochemistry Case II: anatomy, immunology, physiology, microbiology, neuroanatomy
Outcome Variables
Perceived stress-provoking factors among students: a) self-efficacy beliefs, b) faculty and administration, c) workload, d) patient treatment, e) clinical training, f) performance pressure
Callis et al., 2010
Evaluation Tool
Polychro30-item modified DES quesnopoulou tionnaire et al., 2009
Author, Date
Study Design
Table 3. Summary of outcome measures, results, and quality assessment of included studies (whole curriculum intervention)
Studies with concurrent control at different settings
Studies with non-concurrent control at the same setting
Journal of Dental Education ■ Volume 78, Number 1
Weak
Weak
Weak
Weak
Weak
Weak
Study Quality
January 2014 ■ Journal of Dental Education
40-item true/false questionnaire 2. NSD
2. Score of students using negative marking
Oral comprehensive exam (OCE)
Login et al., 1997
4. Percentage of graduates entering postdoctoral programs
4. School records
1. NSD 2. PBL higher in (e)**; NSD in other components
2. Students’ test score in each component: a) organization and thoroughness, b) diagnosis, c) primary treatment pan, d) alternate treatment plan, e) science and medical knowledge, f) dental knowledge
Higher mean scores in PBL**
4. Higher in PBL*
3. Higher in PBL**
2. Lower in PBL**
1. Higher mean scores in PBL**
2. NSD
1. Higher in PBL on midterm** and final* OSCEs
1. Total students’ test scores
Students’ test scores
3. Percentage of students graduating on time
3. School records
NBDE part 1
2. Attrition rate
2. School records
Shuler et al., 1998
1. Students’ test scores
1. NBDE part 1
2. Clinical examination scores
2. Periodontics performance evaluation
Susarla et al., 2003
1. Scores of two preclinical examinations (midyear, final)
1. OSCEs
Rich et al., 2005
Higher mean scores in PBL*
1st-yr students rated PBL higher in (c)**, (d)**, (f)*, (j)*, (k)**; LBS higher in (b)**, (h)*; NSD in other items. 2nd-yr students rated PBL higher in (c)**, (d)**; LBS higher in (b)**; NSD in other items. 3rd-yr students rated PBL higher in (c)**, (j)**; LBS higher in (b)*, (g)*, (h)*; NSD in other items.
3. 11 statements regarding curriculum: a) learn from other students, b) too many topics, c) enough time to understand, d) aims are clear, e) students are friendly, f) developing independence, g) unclear assessment criteria, h) does not encourage thinking, i) consistent advice, j) staff supportive, k) learning to self-assess Students’ test scores
2. 1st-yr students stated balance was better in PBL*; NSD for 2nd- and 3rd-yr students.
2. Balance between theory and application
NBDE part 1
1. 1st-yr, 2nd-yr, and 3rd-yr students stated workload was heavier in LBS*.
1. Workload of course
Students’ perception of curriculum:
1. PBL students scored higher**
1. Score of students using positive marking
Recall of knowledge in anatomy, biochemistry, oral biology, and physiology:
3. NSD
3. VAS questions on graduates’ feelings about how undergraduate studies affected their ability to practice, communicate with patients, and keep up-to-date.
Sukotjo et al., 2007
Townsend Five-point rating scale queset al., 1997 tionnaire
Last et al., 2001
2. NSD
2. Up-to-datedness in the latest practice protocols
Moderate
Moderate
Moderate
Weak
Moderate
Weak
Weak
*Significant difference (p<0.05) **Significant difference (p<0.01)
CDE=continuing dental education; CT=controlled trial (non-randomized); DES=dental environment stress; LBS=lecture-based system; NBDE=National Board Dental Examination; NSD=no significant difference; OSCE=objective structured clinical examination; PBL=problem-based learning; VAS=visual analog scale
Retrospective cohorts
107
REFERENCES
1. Fincham AG, Shuler CF. The changing face of dental education: the impact of PBL. J Dent Educ 2001;65(5): 406-21. 2. Use of problem-based learning in Canadian and U.S. dental schools: results of a survey. J Can Dent Assoc 2002;68(1):26. 3. Berkson L. Problem-based learning: have the expectations been met? Acad Med 1993;68(10 Suppl):S79-88. 4. Barrows HS. Problem-based learning in medicine and beyond: a brief overview. New Directions Teach Learn 1996;68:3-12. 5. Barrows HS. The essentials of problem-based learning. J Dent Educ 1998;62(9):630-3. 6. Hartling L, Spooner C, Tjosvold L, Oswald A. Problembased learning in preclinical medical education: 22 years of outcome research. Med Teach 2010;32(1):28-35. 7. Neufeld VR, Barrows HS. The “McMaster philosophy”: an approach to medical education. J Med Educ 1974;49(11):1040-50. 8. Rohlin M, Petersson K, Svensater G. The Malmo model: a problem-based learning curriculum in undergraduate dental education. Eur J Dent Educ 1998;2(3):103-14. 9. Kassebaum DK, Hendricson WD, Taft T, Haden NK. The dental curriculum at North American dental institutions in 2002-03: a survey of current structure, recent innovations, and planned changes. J Dent Educ 2004;68(9):914-31. 10. Haden NK, Hendricson WD, Kassebaum DK, et al. Curriculum change in dental education, 2003-09. J Dent Educ 2010;74(5):539-57. 11. Vernon DT, Blake RL. Does problem-based learning work? A meta-analysis of evaluative research. Acad Med 1993;68(7):550-63. 12. Albanese MA, Mitchell S. Problem-based learning: a review of literature on its outcomes and implementation issues. Acad Med 1993;68(1):52-81. 13. Smits PB, Verbeek JH, de Buisonje CD. Problem-based learning in continuing medical education: a review of controlled evaluation studies. BMJ 2002;324(7330): 153-6. 14. Koh GC, Khoo HE, Wong ML, Koh D. The effects of problem-based learning during medical school on physician competence: a systematic review. CMAJ 2008;178(1):34-41. 15. Polyzois I, Claffey N, Mattheos N. Problem-based learning in academic health education: a systematic literature review. Eur J Dent Educ 2010;14(1):55-64. 16. Holmboe ES. Faculty and the observation of trainees’ clinical skills: problems and opportunities. Acad Med 2004; 79(1):16-22. 17. Suksudaj N, Townsend GC, Kaidonis J, et al. Acquiring psychomotor skills in operative dentistry: do innate ability and motivation matter? Eur J Dent Educ 2012;16(1): e187-94. 18. Rich SK, Keim RG, Shuler CF. Problem-based learning versus a traditional educational methodology: a comparison of preclinical and clinical periodontics performance. J Dent Educ 2005;69(6):649-62. 19. Thomas BH, Ciliska D, Dobbins M, Micucci S. A process for systematically reviewing the literature: providing the
108
research evidence for public health nursing interventions. Worldviews Evid Based Nurs 2004;1(3):176-84. 20. Deeks JJ, Dinnes J, D’Amico R, et al. Evaluating nonrandomized intervention studies. Health Technol Assess 2003;7(27):iii-x,1-173. 21. Jackson N, Waters E. Criteria for the systematic review of health promotion and public health interventions. Health Promot Int 2005;20(4):367-74. 22. Armijo-Olivo S, Stiles CR, Hagen NA, et al. Assessment of study quality for systematic reviews: a comparison of the Cochrane Collaboration Risk of Bias Tool and the Effective Public Health Practice Project Quality Assessment Tool—methodological research. J Eval Clin Pract 2012;18(1):12-8. 23. Tu MG, Yu CH, Wu LT, et al. Dental and medical students’ perspectives on early exposure to PBL in Taiwan. J Dent Educ 2012;76(6):746-51. 24. Pardamean B. Measuring change in critical thinking skills of dental students educated in a PBL curriculum. J Dent Educ 2012;76(4):443-53. 25. McHarg J, Kay EJ, Coombes LR. Students’ engagement with their group in a problem-based learning curriculum. Eur J Dent Educ 2012;16(1):e106-10. 26. Marshall TA, Finkelstein MW, Qian F. Improved student performance following instructional changes in a problembased learning curriculum. J Dent Educ 2011;75(4): 466-71. 27. Takkunen M, Turpeinen H, Viisanen H, et al. Introduction of real patients into problem-based learning in preclinical first-year anatomy curriculum. Med Teach 2011;33(10):854-6. 28. Qin XJ, Kong J, Lu L, et al. Application of problem-based learning in a large class in stomatology course. J Oral Maxillofac Surg 2010;68(4):739-43. 29. Haak R, Rosenbohm J, Koerfer A, et al. The effect of undergraduate education in communication skills: a randomized controlled clinical trial. Eur J Dent Educ 2008;12(4):213-8. 30. Sukotjo C, Thammasitboon K, Howell H, Karimbux N. Students’ perceptions of prosthodontics in a PBL hybrid curriculum. J Prosthodont 2008;17(6):495-501. 31. Ratzmann A, Wiesmann U, Gedrange T, Kordass B. Early patient contact in undergraduate dental education in Germany: “The Greifswald Model.” Eur J Dent Educ 2007;11(2):93-8. 32. Haghparast N, Sedghizadeh PP, Shuler CF, et al. Evaluation of student and faculty perceptions of the PBL curriculum at two dental schools from a student perspective: a cross-sectional survey. Eur J Dent Educ 2007;11(1):14-22. 33. Kieser J, Herbison P, Harland T. The influence of context on students’ approaches to learning: a case study. Eur J Dent Educ 2005;9(4):150-6. 34. Koufogiannakis D, Buckingham J, Alibhai A, Rayner D. Impact of librarians in first-year medical and dental student problem-based learning (PBL) groups: a controlled study. Health Info Libr J 2005;22(3):189-95. 35. Rafeek RN, Marchan SM, Naidu RS, Carrotte PV. Perceived competence at graduation among dental alumni of the University of the West Indies. J Dent Educ 2004;68(1):81-8. 36. Susarla SM, Bergman AV, Howell TH, Karimbux NY. Problem-based learning and research at the Harvard
Journal of Dental Education ■ Volume 78, Number 1
School of Dental Medicine: a ten-year follow-up. J Dent Educ 2004;68(1):71-6. 37. Ferguson MB, Sobel M, Niederman R. Preclinical restorative training. J Dent Educ 2002;66(10):1159-62. 38. Lohman MC, Finkelstein M. Designing cases in problembased learning to foster problem-solving skill. Eur J Dent Educ 2002;6(3):121-7. 39. Lennon AM, Anderson PF, McDonald JL, Stookey GK. Problem-based learning and the dental school library. J Dent Educ 2001;65(11):1219-24. 40. Chen SK, Chang HF, Chiang CP. Group learning factors in a problem-based course in oral radiology. Dentomaxillofac Radiol 2001;30(2):84-7. 41. Lohman M, Finkelstein M. Designing groups in problembased learning to promote problem-solving skill and self-directedness. Instructional Sci 2000;28(4):291-307. 42. Fincham AG, Baehner R, Chai Y, et al. Problem-based learning at the University of Southern California School of Dentistry. J Dent Educ 1997;61(5):417-25. 43. Obrez A, Lee DJ, Organ-Boshes A, et al. A clinically oriented complete denture program for second-year dental students. J Dent Educ 2009;73(10):1194-201. 44. Kindelan J, Roberts-Harry D, Luther F. A study to determine if changes to an undergraduate orthodontic course improved course quality and student satisfaction. Eur J Dent Educ 1997;1(3):138-42. 45. Petersson K, Olsson H, Soderstrom C, et al. Undergraduate education in endodontology at two European dental schools: a comparison between the Faculty of Odontology, Malmo University, Malmo, Sweden and Faculty of Odontology, Paris 5 University (Rene Descartes), France. Eur J Dent Educ 2002;6(4):176-81. 46. Matlin KS, Libert E, McArdle PJ, Howell TH. Implementing the problem-based curriculum at Harvard School of Dental Medicine. J Dent Educ 1998;62(9):693-708. 47. Wetherell J, Mullins G, Winning T, Townsend G. Firstyear responses to a new problem-based curriculum in dentistry. Aust Dent J 1996;41(5):351-4. 48. Zhang Y, Chen G, Fang X, et al. Problem-based learning in oral and maxillofacial surgery education: the Shanghai hybrid. J Oral Maxillofac Surg 2012;70(1):e7-e11. 49. Yiu CK, McGrath C, Bridges S, et al. Graduates’ perceived preparedness for dental practice from PBL and traditional curricula. J Dent Educ 2011;75(9):1270-9. 50. Callis AN, McCann AL, Schneiderman ED, et al. Application of basic science to clinical problems: traditional vs. hybrid problem-based learning. J Dent Educ 2010; 74(10):1113-24. 51. Polyzois I, Claffey N, Attstrom R, et al. The role of the curriculum and other factors in determining the mediumto long-term attitude of the practicing dentist towards lifelong learning. Eur J Dent Educ 2010;14(2):84-91. 52. Moreno-Lopez LA, Somacarrera-Perez ML, Diaz-Rodriguez MM, et al. Problem-based learning versus lectures:
January 2014 ■ Journal of Dental Education
comparison of academic results and time devoted by teachers in a course on dentistry in special patients. Med Oral Patol Oral Cir Bucal 2009;14(11):e583-7. 53. Polychronopoulou A, Divaris K. Dental students’ perceived sources of stress: a multi-country study. J Dent Educ 2009;73(5):631-9. 54. Thammasitboon K, Sukotjo C, Howell H, Karimbux N. Problem-based learning at the Harvard School of Dental Medicine: self-assessment of performance in postdoctoral training. J Dent Educ 2007;71(8):1080-9. 55. Sukotjo C, Thammasitboon K, Howell H, Karimbux N. The impact of targeted shortened preclinical exercises on student perceptions and outcomes. J Dent Educ 2007;71(8):1070-9. 56. Reich S, Simon JF, Ruedinger D, et al. Evaluation of two different teaching concepts in dentistry using computer technology. Adv Health Sci Educ Theory Pract 2007;12(3):321-9. 57. Susarla SM, Medina-Martinez N, Howell TH, Karimbux NY. Problem-based learning: effects on standard outcomes. J Dent Educ 2003;67(9):1003-10. 58. Last KS, Appleton J, Stevenson H. Basic science knowledge of dental students on conventional and problembased learning (PBL) courses at Liverpool. Eur J Dent Educ 2001;5(4):148-54. 59. Greenwood LF, Townsend GC, Wetherell JD, Mullins GA. Self-perceived competence at graduation: a comparison of dental graduates from the Adelaide PBL curriculum and the Toronto traditional curriculum. Eur J Dent Educ 1999;3(4):153-8. 60. Shuler CF, Fincham AG. Comparative achievement on National Dental Board Examination Part I between dental students in problem-based learning and traditional educational tracks. J Dent Educ 1998;62(9):666-70. 61. Login GR, Ransil BJ, Meyer M, et al. Assessment of preclinical problem-based learning versus lecture-based learning. J Dent Educ 1997;61(6):473-9. 62. Walton JN, Clark DC, Glick N. An outcomes assessment of a hybrid-PBL course in treatment planning. J Dent Educ 1997;61(4):361-7. 63. Townsend GC, Winning TA, Wetherell JD, Mullins GA. New PBL dental curriculum at the University of Adelaide. J Dent Educ 1997;61(4):374-87. 64. Dochy F, Segers M, Van den Bossche P, Gijbels D. Effects of problem-based learning: a meta-analysis. Learn Instr 2003;13(5):533-68. 65. Colliver JA. Effectiveness of problem-based learning curricula: research and theory. Acad Med 2000;75(3):259-66. 66. Winning T, Townsend G. Problem-based learning in dental education: what’s the evidence for and against . . . and is it worth the effort? Aust Dent J 2007;52(1):2-9. 67. Eva KW. Issues to consider when planning and conducting educational research. J Dent Educ 2004;68(3):316-23.
109
