|Year : 2020 | Volume
| Issue : 1 | Page : 10-14
Assessment of problem-based learning sessions in undergraduate dental students
Hassan Mohamed Abouelkheir
Department of Oral and Maxillofacial Surgery and Diagnostic Sciences, College of Dentistry, Riyadh ELM University, Riyadh, Kingdom of Saudi Arabia
|Date of Submission||27-Dec-2019|
|Date of Decision||06-Jan-2020|
|Date of Acceptance||28-Jan-2020|
|Date of Web Publication||28-Mar-2020|
Hassan Mohamed Abouelkheir
Abouelkheir, College of Dentistry, Riyadh ELM University, P.O. Box 84891, Riyadh 11681
Kingdom of Saudi Arabia
Source of Support: None, Conflict of Interest: None
Objectives: The aim of the present study is to assess the student's performance in problembased learning (PBL) and its relation to other assessment forms such as multiplechoice questions (MCQs) and standardized oral examination. Materials and Methods: Eighty-two undergraduate dental students were participated in PBL sessions who were evaluated through a customized rubric. Then, PBL sessions were compared with the final written MCQ examination as well as a final standardized oral examination. Results: There was no statistical significance between PBL and final written examination (F = 308) and between PBL and oral examination (F = 4.667). There was a high correlation between the F-written and final oral examination (r = 0.708) and the average correlation between PBL and F-written MCQs (r = 0.436) and between PBL and oral examination (r = 0.423). Conclusion: Self-directed learning (SDL) is a key element in PBL. Different continuous assessment tests such as PBL, MCQs, and oral examinations are needed to assess different competencies such as knowledge, problem-solving, clinical skills, and SDL; No one test can assess all types of Competencies. Interpersonal communication and communication skills need further research for better assessment.
Keywords: Assessment, dental students, multiple-choice questions, oral examination, problem-based learning
|How to cite this article:|
Abouelkheir HM. Assessment of problem-based learning sessions in undergraduate dental students. J Dent Res Rev 2020;7:10-4
|How to cite this URL:|
Abouelkheir HM. Assessment of problem-based learning sessions in undergraduate dental students. J Dent Res Rev [serial online] 2020 [cited 2020 Jun 5];7:10-4. Available from: http://www.jdrr.org/text.asp?2020/7/1/10/281511
| Introduction|| |
Problem-based learning (PBL) is an approach to promote self-directed learning (SDL) among a small group of around 6–10 students. PBL should include students, SDL, teachers, problem setting, clinical learning, and resource support. PBL focuses mainly on the ability of students to identify problems, problem solving techniques, and the process of seeking out knowledge and information to answer questions rather than traditional recalling of isolated facts. Therefore, PBL promotes SDL.
Barrows and Tamblyn identify PBL as learning which results from the process of working toward the understanding of or resolution of a problem. One of the earliest users of PBL was the MacMaster Medical school. Since then, many medical schools implemented PBL in the curriculum.
PBL differs from traditional reaching in three categories: first, integrated curriculum that combines basic and clinical sciences and not based on disciplines; second, small group teaching from 5 to 7 students focusing on certain problem; and finally, outcomes which improve knowledge, skills, and attitude.
Outcomes or performance of students regarding knowledge and problem-solving skills can be measured through different continuous assessment tests such as multiple-choice questions (MCQs), short-answer questions, and extended-matching items. Well-constructed MCQs can assess higher-order cognitive knowledge. It can assess remembering and understanding (Level I and II) as well as Level (III and IV) according to Bloom's Taxonomy. Three factors should be kept in mind when constructing MCQs: first, avoid item-writing flaws; second, write MCQs in higher cognitive levels (Level 3 and 4); and finally, analyze MCQs after examination using the facility index (FI) and the discrimination index (DI), where acceptable levels for FI is 26%–74% and DI 0.20–0.30.
Structured oral examinations can be used for measuring outcomes, but they should be standardized by preparing sets of hypothetical cases and suggesting lines of questions and scoring criteria in advance.
Therefore, assessment of PBL should not focus on recall of isolated facts or knowledge, but it should concentrate on the processes and performances in context. Assessment should be consistent with how student learn in PBL settings, match the educational intended learning outcomes of the curriculum. Therefore, PBL assessment should include knowledge and problem-solving skills and professional attitudes. This can be divided into outcome-oriented instruments and process-oriented instruments.
Aim of the research
- To assess the student's performance through final MCQs and final oral examination and to compare the results with PBL (outcome oriented)
- To assess how students fulfill the criteria of PBL during practical sessions (process oriented).
| Materials and Methods|| |
Eighty-two undergraduate dental students of the College of Dentistry in Riyadh ELM University, KSA, were participated in the study. They were divided into two patches (A and B), 41 students in each patch, where the ORAD-424 course had one interactive lecture and one practical session per week. In practical sessions, each patch was divided into small groups (5–7 students), where a selected radiological case was presented by a course director. In practical sessions, three or four groups in each session were formed. Students were fixed to the same groups throughout the course. Students in each group work together to interpret radiographic cases and reach a differential diagnosis for presenting a case. Afterward, a course director (H. M. Abouelkheir) discussed the radiological cases with each student.
Therefore, an assessment was done as follows:
- PBL (process oriented) assessment
- In this study, a practical PBL rubric is implemented to measure three skills: Knowledge acquisition, problem-solving, and critical thinking and personal and interpersonal development and rated at four-point Likert-type scale. It is modified from detailed PBL rubric used by Allareddy et al. [Appendix 1].
Student's performance (outcome based) can be assisted by
- MCQs: Each student can be assisted regarding knowledge and problem-solving through his/her achievements in final written MCQ examination after finishing the course
- Structured oral examination: Each student was assessed through structured standardized final oral examination where hypothetical radiographic cases were prepared after several revisions of cases with students and suggested lines of questions and scoring criteria will be prepared.
Descriptive statistics were done for PBL, final written examination, and final oral examination. One-way ANOVA was done (final written examination, oral examination, and PBL assessment) Finally, Pearson's correlation® among three variables (Fwritten MCQs), oral examination were done. Analysis of data was done using SPSS statistical package version 22 (IBM Corp., Armonk, NY, USA).
The present study was reviewed and approved by the research ethics committee. The institutional review board approval number is RC/IRB/2018/1353.
| Results|| |
[Table 1] shows a descriptive statistic of problembased assessment, final written examination & oral examination in the form of means (M), standard deviation (Std.), and total number of students (N).
|Table 1: Descriptive statistics of F-written (multiple-choice questions), oral examination, and problem-based learning assessment|
Click here to view
There was no significant difference between final-written MCQS examination and final oral examination as well as PBL assessment. All variances were approximately the same [Table 2].
|Table 2: One-way ANOVA among 3 variables (final written, oral examination and PB assessment). All variables were approximate equals and there is no significant difference|
Click here to view
[Table 3] shows that there was a high positive correlation between the F-written MCQs examination and standardized oral examination (0.71%), whereas PBL assessment showed a fair positive correlation with both F-written MCQs examination (0.44%) and oral examination (0.42%).
|Table 3: Pearson's correlation (r) among three variables (F-written (multiple-choice questions), oral examination, and problem-based learning)|
Click here to view
| Discussion|| |
Assessment methods could be conceptualized by looking at five attributes: reliability, validity, educational impact, feasibility, and acceptability. The American Board of Internal Medicine distinguished between four different dimensions of clinical competence. These included abilities (knowledge, technical skills, and interpersonal skills), the problem-solving skill, the nature of medical illness, and social and psychological aspect.
Valle et al. studied the assessment of student performance in PBL tutorial sessions. They performed a 24-item rating scale questionnaire. Items were divided into three categories: independent study, group interaction, and reasoning skills. It was found that there were four factors that affect 76.6% of student performance; independent study, group interaction, reasoning skills and active participation.
PBL gives the student the ability to work as a member of a team and collaborate effectively with others. These skills can be transferred to different situations in the health care team.
In the present study, it was found that there was no significant difference between final written MCQs examination and PBL. At the same time, there was no statistical difference between final oral examination and PBL. Therefore, all assessment methods have limitations and no method can assess all types of knowledge and skills. A good assessment uses a mix of methods depending on the context. It also found that there was a high positive correlation between final written examination and oral examination (0.708), whereas the correlation between PBL and oral examination (0.423) was average and comparable with that correlation with PBL and final written examination (0.436).
Frambach et al. reported about how students' cultural backgrounds impact on PBL. They found that Middle Eastern students expressed more feelings of uncertainty as a cultural factor compared with Dutch and Hong Kong students. Their uncertainty and difficult in adapting to SDL were related to conflict between PBL and their prior educational experiences which depend on teacher-centered secondary education.
A study from Umm Al-Qura University in Saudi Arabia has shown that a lack of proficiency in English language was one of the main difficulties faced by medical students. Kaliyadan et al. also reported that English language proficiency is the most important factor in determining academic performance in the context of summative assessment and less in the context of formative assessment in College of Medicine, King Faisal University. In the present study, students find difficulty in writing radiographic interpretation sheet in PBL. Therefore, more efforts should be made to improve language learning strategies, rather than the time allocated for learning a language.
In the present study, the 3rd part of PBL rubric which was concerned with interpersonal and communication skills and working with others was not clear to dental students and, totally, was teachercentered approach. Evidence-based studies showed that doctors' interpersonal and communication skills have a significant impact on patient care and correlate with improved health outcomes and health-care quality., Therefore, teaching and assessment of competency in interpersonal and communication skills are required at all levels of medical or dental training. Further researches needed to fill this gap for health professionals.
| Conclusion|| |
SDL is a key element in PBL. Different continuous assessment tests such as PBL, MCQs, and oral examinations are needed to assess different competencies such as knowledge, problem-solving, clinical skills, and SDL; No one type of assessment test is capable to assess all competencies. Interpersonal and communication skills need further research for better assessment.
Financial support and sponsorship
Conflicts of interest
There are no conflicts of interest.
| Appendix|| |
College Of Dentistry
ORAD- 424-Gr# / SubGr#
PBL Evaluation chart
| References|| |
Eldredge JD, Teal JB, Ducharme JC, Harris RM, Croghan L, Perea JA. The roles of library liaisons in a problem-based learning. (PBL) Medical school curriculum: A case study from university of new Maxico. Health Libr Eve 1998;15:185-94.
Alexander JG, McDaniel GS, Baldwin MS, Money BJ. Promoting, applying, and evaluating problem-based learning in the undergraduate nursing curriculum. Nurs Educ Perspect 2002;23:248-53.
Barrows HS, Tamblyn RM. In: Problem-Based Learning: An Approach to Medical Education. New York: Springer; 1980.
Swanson DB, Case SM, Van der Vleuten CP. Strategies for student assessment. In: Feletti DB, editors. The Challenge of Problem-Based Learning. New York: St. Martin's Press; 1991. p. 260-73.
Walton HJ, Matthews MB. Essentials of problem-based learning. Med Educ 1989;23:542-58.
Krathwohl D. A revision of bloom's taxonomy: An overview. Theory Pract 2002;41:212-18.
Abouelkheir HM. The criteria and analysis of multiple-choice questions in undergraduate dental examinations. J Dent Res Rev 2018;5:59-64. [Full text]
Swanson DB, Norman GR, Linn R. Performance-based assessment: Lesions from health professions. Educational Researcher 1995;24:5-1,35.
Norman GR. What should be assessed? In: Bode ND, Feletti G, editors. The Challenge of Problem-Based Learning. New York: St. Martin's Press; 1991. p. 254-9.
Neufeld VR, Barrows HS. The “McMaster Philosophy”: An approach to medical education. J Med Educ 1974;49:1040-50.
Allareddy V, Havens AM, Howell TH, Karimbux NY. Evaluation of a new assessment tool in problem-based learning tutorials in dental education. J Dent Educ 2011;75:665-71.
Van Der Vleuten CP. The assessment of professional competence: Developments, research and practical implications. Adv Health Sci Educ Theory Pract 1996;1:41-67.
American Board of Internal Medicine. Clinical competence in internal medicine. Ann Intern MED 1979;90:402-11.
Valle R, Petra L, Martínez-Gonzáez A, Rojas-Ramirez JA, Morales-Lopez S, Piña-Garza B. Assessment of student performance in problem-based learning tutorial sessions. Med Educ 1999;33:818-22.
Carey L, Whittaker KA. Experiences of problem-based learning: Issues for community specialist practitioner students. Nurse Educ Today 2002;22:661-8.
Sood R, Singh T. Assessment in medical education: Evolving perspectives and contemporary trends. Natl Med J India 2012;25:357-64.
Frambach JM, Driessen EW, Chan LC, van der Vleuten CP. Rethinking the globalisation of problem-based learning: How culture challenges self-directed learning. Med Educ 2012;46:738-47.
Almoallim H, Aldahlawi S, Alqahtani E, Alqurashi S, Munshi A. Difficulties facing first-year medical students at Umm Alqura University in Saudi Arabia. East Mediterr Health J 2012;16:1272-7.
Kaliyadan F, Thalamkandathil N, Parupalli SR, Amin TT, Balaha MH, Al Bu Ali WH. English language proficiency and academic performance: A study of a medical preparatory year program in Saudi Arabia. Avicenna J Med 2015;5:140-4.
] [Full text]
Stewart MA. Effective physician-patient communication and health outcomes: A review. CMAJ 1995;152:1423-33.
Stewart M, Brown JB, Boon H, Galajda J, Meredith L, Sangster M. Evidence on patient-doctor communication. Cancer Prev Control 1999;3:25-30.
[Table 1], [Table 2], [Table 3]