Abstract
The advent of artificial intelligence (AI) and natural language processing technologies has ushered in a new era of innovative educational tools and resources. ChatGPT, a prominent AI-powered language model, has demonstrated remarkable potential in various fields. In this study, we aim to explore the ability of ChatGPT to answer Pharmacotherapy Specialist (BCPS) exam questions and provide justifications for its answers. We conducted a comprehensive content analysis to scrutinize the applicability and competence of ChatGPT in addressing BCPS exam inquiries while substantiating the correctness of its responses. Our findings reveal that ChatGPT was able to answer 94 out of 128 questions (73.43%) correctly and justify 80.16% of the correct answers appropriately and 76.18% of the noncorrect answers in all questions. Conclusion: The ChatGPT’s ability to answer the Pharmacotherapy Specialist (BCPS) exam and justify the answers was good. ChatGPT could help pharmacists in their preparation for this exam with caution. ChatGPT is still in the early phase of use by educators and students worldwide, and its ability to answer exams will be better in the near future. Attending training workshops about ChatGPT and AI is very important and highly recommended. Practice ChatGPT in medical and health sciences education is very important and highly recommended to explore the potential uses, benefits and risks and suggest recommendations for best practices.
Keywords
Artificial Intelligence (AI) ChatGPT Pharmacotherapy Specialist (BCPS) exam Educational Tool Natural Language Processing
References
- [1] Masters K. Artificial intelligence in medical education. Med Teach. 2019; 41(9):976-980. https://doi.org/10.1080/0142159x.2019.1595557.
- [2] Loeckx J. Blurring boundaries in education: Context and impact of MOOCs. Int Rev Res Open Distrib Learn. 2016; 17(3):92-121. https://doi.org/10.19173/irrodl.v17i3.2395.
- [3] Ahuja AS. The impact of artificial intelligence in medicine on the future role of the physician. PeerJ. 2019; 7:e7702. https://doi.org/10.7717/peerj.7702.
- [4] Bohr A, Memarzadeh K. The rise of artificial intelligence in healthcare applications. In: Artificial Intelligence in Healthcare; 2020. p. 25-60. Academic Press. https://doi.org/10.1016/B978-0-12-818438-7.00002-2.
- [5] Secinaro S, Calandra D, Secinaro A, Muthurangu V, Bilancone C. The role of artificial intelligence in healthcare: a structured literature review. BMC Med Inform Decis Mak. 2021; 21:1-23. https://doi.org/10.1186/s12911-021-01488-9.
- [6] Davenport TH, Ronanki R. Artificial intelligence for the real world. Harv Bus Rev. 2018;96(1):108-116. Available from: https://hbr.org/2018/01/artificial-intelligence-for-the-real-world. Accessed 27 December 2023.
- [7] Hosny A, Parmar C, Quackenbush J, Schwartz LH, Aerts HJ. Artificial intelligence in radiology. Nat Rev Cancer. 2018;18(8):500-510. https://doi.org/10.1038/s41568-018-0016-5.
- [8] Roll I, Wylie R. Evolution and revolution in artificial intelligence in education. Int J Artif Intell Educ. 2016;26:582-599. https://doi.org/10.1007/s40593-016-0110-3.
- [9] McCarthy J. What is artificial intelligence. 2007. Available from: https://cse.unl.edu/~choueiry/S09-476-876/Documents/whatisai.pdf. Accessed 27 December 2023.
- [10] Shapiro SC. John Wiley & Sons. Encyclopedia of Artificial Intelligence, Volume 1, Second Edition. New Jersey: A Wiley Interscience Publication; 1992.
- [11] OPEN AI. Introducing Chat GPT. Available from: https://openai.com/blog/chatgpt/. Accessed 27 December 2023.
- [12] Al-Worafi YM, Hermansyah A, Tan CS, Choo CY, Bouyahya A, Paneerselvam GS, Liew KB, Goh KW, Ming LC. Applications, benefits, and risks of ChatGPT in medical and health sciences research: An experimental study. Prog Microbes Mol Biol. 2023;6(1):a0000337. https://doi.org/10.36877/pmmb.a0000337.
- [13] Wong R, Ming L, Raja Ali RA. The intersection of ChatGPT, clinical medicine, and medical education. JMIR Med Educ. 2023;9:e47274. https://doi.org/10.2196/47274.
- [14] Board of Pharmacists Specialists (BPS). About BPS. Available from: https://www.bpsweb.org/about-bps/history/. Accessed 27 December 2023.
- [15] Kung TH, Cheatham M, Medenilla A, Sillos C, De Leon L, Elepaño C, Madriaga M, Aggabao R, Diaz-Candido G, Maningo J, Tseng V. Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models. PLOS Digit Health. 2023;2(2):e0000198. https://doi.org/10.1371/journal.pdig.0000198
- [16] American College of Clinical Pharmacy (ACCP). The Updates in Therapeutics®: Pharmacotherapy Preparatory Review and Recertification Course. Available from: https://www.accp.com/store/product.aspx?pc=PPC21G. Accessed 27 December 2023.
- [17] American College of Clinical Pharmacy (ACCP). 2022 ACCP Pharmacotherapy Mock Exam online product. Available from: https://www.accp.com/store/product.aspx?pc=MEPH22. Accessed 27 December 2023.
Details
| Primary Language | English |
|---|---|
| Subjects | Clinical Pharmacology and Therapeutics |
| Journal Section | Articles |
| Authors | |
| Publication Date | June 28, 2025 |
| Submission Date | October 11, 2023 |
| Acceptance Date | January 23, 2024 |
| Published in Issue | Year 2024 Volume: 28 Issue: 3 |