Review
BibTex RIS Cite

SÖZEL İLETİŞİM KURAMAYAN YETİŞKİN YOĞUN BAKIM HASTALARINDA KULLANILAN AĞRI ÖLÇEKLERİNİN KARŞILAŞTIRILMASI: SİSTEMATİK BİR LİTERATÜR İNCELEMESİ

Year 2025, Volume: 29 Issue: 1, 67 - 80, 30.04.2025

Sevilay Erden , Yeşim Yaman Aktaş , Pınar Kaya

https://doi.org/10.62111/ybhd.1477517

Abstract

ÖZET
Amaç
Bu çalışmanın amacı, ağrıyı değerlendirmek için rutin olarak kullanılan Davranışsal Ağrı Ölçeği, Kritik Bakım Ağrı Gözlem Aracı ve Sözel Olmayan Ağrı Ölçeği'ni değerlendiren literatürü doğrulamaktır.
Yöntemler
Bu çalışma metodolojik bir sistematik derlemedir. Bu çalışmada, MEDLINE, OVID, Google Scholar, Science Direct, Scopus ve PubMed veritabanlarında 2010-2024 (Nisan) arasındaki çalışmalar kullanıldı. Bu çalışmada uygunluk kriterlerini tanımlamak için anımsatıcı PICOS yöntemi kullanıldı. Veriler çıkarıldı, özetlendi ve yayınlanan uyumluluk ve kullanılabilirlik sınıflandırmaları kullanılarak değerlendirildi.
Bulgular
Araştırmalar, Davranışsal Ağrı Ölçeği, Kritik Bakım Ağrı Gözlem Aracı ve Sözel Olmayan Ağrı Ölçeği'nin sözel olarak iletişim kuramayan hastalarda ağrıyı değerlendirmek için geçerli ve güvenilir ölçekler olduğunu doğruladı. Ağrı değerlendirme araçları, iki çalışma hariç, uyumluluk ve kullanılabilirlik açısından benzerdi.
Sonuçlar
Düşük hemodinamik riskli kritik bakım hastaları için değerlendirilen tüm ağrı değerlendirme araçları arasından Davranışsal Ağrı Ölçeği ve Kritik Bakım Ağrı Gözlem Aracı en uygun bulundu. Kritik bakım ortamında uygun bir ağrı değerlendirme aracının sistematik ve tutarlı kullanımı, ağrı kontrolünü optimize eder ve iyileştirir.

Keywords

kritik hasta yoğun bakım sözsüz iletişim ağrı değerlendirmesi

Supporting Institution

Destekleyen kurum/kuruluş bulunmamaktadır.

References

  • 1. Ada M, Yılmaz E. Pain behaviors in patients under mechanical ventilatory support during intravascular interventions. Kocaeli Üniversitesi Sağlık Bilimleri Dergisi. 2020;6(2):138-144.
  • 2. Efe KA, Çaydam ÖD. Assessment of pain behaviour among patients submitted to mechanical ventilation in Intensive Care Unit. Avrasya Sağlık Bilimleri Dergisi. 2020;3(1):23-34.
  • 3. Erden S. Yoğun bakımda ağrı yönetiminde hemşirenin anahtar rolleri. Van Tıp Dergisi. 2015;22(4):332-336.
  • 4. Kaya P, Erden S. Cross-cultural adaptation, validity and reliability of the Turkish version of Revised Nonverbal Pain Scale. Agri. 2019; 31(1):15-22.
  • 5. Yaman Aktaş Y, Karabulut N. Relief of procedural pain in critically ill patients by music therapy: A randomized controlled trial. BMC Complement Med Ther. 2019;26(3):156-165.
  • 6. Candan Dönmez Y, Demir Korkmaz F, Geçit S. Yoğun bakım ünitesindeki çevresel stresörlerin hastalar tarafından algılanması. Türkiye Klinikleri Hemşirelik Bilimleri Dergisi. 2020;12(2):190-197.
  • 7. Çelik S. Yoğun bakım hastalarında ağrı yönetimi. Yoğun Bakım Hemşireliği Dergisi. 2016;20(1):1-8.
  • 8. Topolovec-Vranic J, Canzian S, Innis J, et al. Patient satisfaction and documentation of pain assessments and management after implementing the adult nonverbal pain scale. Am J Crit Care. 2010;19(4):345-354.
  • 9. Yaman Aktaş Y, Karabulut N. The use of cold therapy, music therapy and lidocaine spray for reducing pain and anxiety following chest tube removal. BMC Complement Med Ther. 2019;34:179-184.
  • 10. Herr K, Coyne PJ, Ely E, et al. Pain assessment in the patient unable to self-report: Clinical practice recommendations in support of the ASPMN 2019 position statement. Pain Manag Nurs. 2019;20(5):404-417.
  • 11. Arbour C, Gélinas C. Are vital signs valid indicators for the assessment of pain in postoperative cardiac surgery ICU adults? Intensive Crit Care Nurs. 2010;26(2):83-90.
  • 12. Yaman Aktaş Y, Karabulut N. A Turkish version of the Critical-Care Pain Observation Tool: reliability and validity assessment. J Perianesth Nurs. 2017;32(4):341-351.
  • 13. Gélinas C. Pain assessment in the critically ill adult: Recent evidence and new trends. Intensive Crit Care Nurs. 2016;34:1-11.
  • 14. Herr K, Coyne PJ, McCaffery M, et al. Pain assessment in the patient unable to self-report: Position statement with clinical practice recommendations. Pain Manag Nurs. 2011;12:230-250.
  • 15. Erden S, Demir N, Ugras GA, et al. Vital signs: Valid indicators to assess pain in intensive care unit patients? An observational, descriptive study. Nurs Health Sci. 2018;20(4):502-508.
  • 16. Relland LM, Gehred A, Maitre NL. Behavioral and physiological signs for pain assessment in preterm and term neonates during a nociception-specific response: A systematic review. J Pediatr Neurol. 2019;90:13-23. 17. Hora TCNSD, Alves IGN. Scales for the assessment of pain in the intensive care unit. Systematic review. BrJP. São Paulo, 2020; 3(3):63-74.
  • 18. Durán-Crane A, Laserna A, López-Olivo MA, et al. Clinical practice guidelines and consensus statements about pain management in critically ill end-of-life patients: A systematic review. J Crit Care Med. 2019; 47(11):1619-1626.
  • 19. Al Darwish ZQ, Hamdi R, Fallatah S. Evaluation of pain assessment tools in patients receiving mechanical ventilation. AACN Adv Crit Care. 2016;27(2):162-172.
  • 20. Chanques G, Pohlman A, Kress JP, et al. Psychometric comparison of three behavioural scales for the assessment of pain in critically ill patients unable to self-report. Crit Care. 2014;18(5):1-12.
  • 21. PRISMA 2020 Checklist. Accessed 13 September 2022. https://prisma-statement.org/documents/PRISMA_2020_checklist.pdf
  • 22. Joanna Briggs Institute, JBI Levels of Evidence 2014. Accessed 13 September 2022 https://jbi.global/sites/default/files/2019-05/JBI-Levels-of-evidence_2014_0.pdf
  • 23. Darmanto D, Haloho AB, Zainal R, et al. Suitability of CPOT and BPS to Assess Pain Response in Intubated Mohammad Hoesin Hospital Intensive Care Patients. Journal of Anesthesiology and Clinical Research. 2020;1(2):85-94.
  • 24. Gomarverdi S, Sedighie L, Seifrabiei MA, et al. Comparison of two pain scales: Behavioral pain scale and critical-care pain observation tool during invasive and noninvasive procedures in intensive care unit-admitted patients. Iran J Nurs Midwifery Res. 2019;24(2):151.
  • 25. Ito Y, Teruya K, Nakajima E. Evaluation of pain severity in critically ill patients on mechanical ventilation. Intensive Crit Care Nurs. 2022;68:103118.
  • 26. Juarez P, Bach A, Baker M, et al. Comparison of two pain scales for the assessment of pain in the ventilated adult patient. Dimens Crit Care Nurs. 2010;29(6):307-315.
  • 27. Klein C, Caumo W, Gélinas C, et al. Validation of two pain assessment tools using a standardized nociceptive stimulation in critically ill adults. J Pain Symptom Manage. 2018;56(4):594-601.
  • 28. Kontou P, Kotoulas SC, Kalliontzis S, et al. Evaluation of pain scales and outcome in critically ill patients of a Greek ICU. J Pain Palliat Care Pharmacother. 2022;1-10.
  • 29. Liu Y, Li L, Herr K. Evaluation of two observational pain assessment tools in Chinese critically ill patients. Pain Med. 2015;16(8):1622-1628.
  • 30. Nazari R, Froelicher ES, Nia HS, et al. Diagnostic values of the Critical Care Pain Observation Tool and the Behavioral Pain Scale for pain assessment among unconscious patients: A comparative study. Indian J Crit Care Med. 2022;26(4):472.
  • 31. Rijkenberg S, Stilma W, Endeman H, et al. Pain measurement in mechanically ventilated critically ill patients: Behavioral pain scale versus critical-care pain observation tool. Crit Care. 2015;30(1):167-172.
  • 32. Rijkenberg S, Stilma W, Bosman RJ, et al. Pain measurement in mechanically ventilated patients after cardiac surgery: Comparison of the Behavioral Pain Scale (BPS) and the Critical-Care Pain Observation Tool (CPOT). J Cardiothorac Vasc Anesth. 2017;31(4):1227-1234.
  • 33. Severgnini P, Pelosi P, Contino E, et al. Accuracy of Critical Care Pain Observation Tool and Behavioral Pain Scale to assess pain in critically ill conscious and unconscious patients: Prospective, observational study. J Intensive Care. 2016;4(1):1-8.
  • 34. Wojnar-Gruszka K, Sega A, Płaszewska-Żywko L, et al. Pain assessment with the BPS and CCPOT behavioral pain scales in mechanically ventilated patients requiring analgesia and sedation. International Journal of Environmental Research and Public Health. 2022;19(17):10894.
  • 35. Waladani B, Nazhifah IYESN, Yunita RI. Application of pain scale assessment in patients attached mechanical ventilator in Intensive Care Unit. In 4th International Conference on Sustainable Innovation 2020–Health Science and Nursing (ICoSIHSN 2020) 2021:332-336. Atlantis Press.
  • 36. Bahramnezhad F, Salamat E, Sharifi F, et al. Comparing pain intensity of two instruments in predicting the outcomes of patients under mechanical ventilation admitted to intensive care units. Indian Journal of Pain. 2023;37(1):15-21.
  • 37. Fröhlich MR, Meyer G, Spirig R, et al. Comparison of the Zurich observation pain assessment with the Behavioural Pain Scale and the Critical Care Pain Observation Tool in nonverbal patients in the intensive care unit: A prospective observational study. Intensive and Critical Care Nursing. 2020;60,102874.
  • 38. Gelinas C, Boitor M, Puntillo KA, et al. Behaviors indicative of pain in brain injured adult patients with different levels of consciousness in the intensive care unit. J Pain Symptom Manage. 2019;57(4):761-773.
  • 39. Kotfis K, Zegan-Barańska M, Szydłowski Ł, et al. Methods of pain assessment in adult intensive care unit patients—Polish version of the CPOT (Critical Care Pain Observation Tool) and BPS (Behavioral Pain Scale). Anaesthesiol Intensive Ther. 2017;49(1):66-72.
  • 40. Puntillo K, Gélinas C, Chanques G. Next steps in ICU pain research. Intensive Care Med. 2017;43(9):1386-1388.
  • 41. Vadelka A, Busnelli A, Bonetti L. Comparison between two behavioural scales for the evaluation of pain in critical patients, as related to the state of sedation: An observational study. Scenario. 2017;34(2):4-14.
  • 42. Kawagoe CK, Matuoka JY, Salvetti MG. Instrumentos de avaliação da dor em pacientes críticos com dificuldade de comunicação verbal: revisão de escopo. Revista Dor. 2017;18(2):161-165.
  • 43. Zhai Y, Cai S, Zhang Y. The diagnostic accuracy of critical care pain observation tool (CPOT) in ICU patients: A systematic review and meta-analysis. J Pain Symptom Manage. 2020;60(4):847–856.
  • 44. Wøien H. Movements and trends in intensive care pain treatment and sedation: What matters to the patient? J Clin Nurs. 2020;29(7-8):1129-1140.
  • 45. Faust AC, Rajan P, Sheperd LA, et al. Impact of an analgesia-based sedation protocol on mechanically ventilated patients in a medical intensive care unit. Anesth Analg. 2016;123(4): 903.
  • 46. Bardwell J, Brimmer S, Davis W. Implementing the ABCDE Bundle, Critical-Care Pain Observation Tool, and Richmond Agitation-Sedation Scale to reduce ventilation time. AACN Adv Crit Care. 2020; 31(1):16-21.

COMPARISON OF PAIN SCALES USED IN ADULT NON-VERBAL CRITICAL CARE PATIENTS: A SYSTEMATIC LITERATURE REVIEW

Year 2025, Volume: 29 Issue: 1, 67 - 80, 30.04.2025

Sevilay Erden , Yeşim Yaman Aktaş , Pınar Kaya

https://doi.org/10.62111/ybhd.1477517

Abstract

ABSTRACT
Purpose
The purpose of this study was to verify the literature evaluating the Behavioral Pain Scale, the Critical Care Pain Observation Tool, and the Nonverbal Pain Scale, which are routinely used to assess pain in the patient.
Methods
This study was a methodological systematic review. Herein, MEDLINE, OVID, Google Scholar, Science Direct, Scopus and PubMed databases were used from 2010 to 2024 (April). The mnemonic PICOS was used to define the eligibility criteria in this study. Data were extracted, summarized, evaluated using published compatibility and usability classifications.
Findings
Research has verified that the Behavioral Pain Scale, the Critical Care Pain Observation Tool, and the Nonverbal Pain Scale are legitimate and reliable scales for assessing pain in patients who cannot communicate verbally. Pain assessment tools were similar in terms of compatibility and usability, with the exception of two studies.
Conclusions
Of all the pain assessment units considered, Behavioral Pain Scale and the Critical Care Pain Observation Tool were found to be most appropriate for critical care patients at hemodynamic risk. The systematic and consistent use of an appropriate pain assessment tool in the critical care setting optimizes and improves pain control.

Keywords

Critically ill intensive care nonverbal communication pain assessment

References

  • 1. Ada M, Yılmaz E. Pain behaviors in patients under mechanical ventilatory support during intravascular interventions. Kocaeli Üniversitesi Sağlık Bilimleri Dergisi. 2020;6(2):138-144.
  • 2. Efe KA, Çaydam ÖD. Assessment of pain behaviour among patients submitted to mechanical ventilation in Intensive Care Unit. Avrasya Sağlık Bilimleri Dergisi. 2020;3(1):23-34.
  • 3. Erden S. Yoğun bakımda ağrı yönetiminde hemşirenin anahtar rolleri. Van Tıp Dergisi. 2015;22(4):332-336.
  • 4. Kaya P, Erden S. Cross-cultural adaptation, validity and reliability of the Turkish version of Revised Nonverbal Pain Scale. Agri. 2019; 31(1):15-22.
  • 5. Yaman Aktaş Y, Karabulut N. Relief of procedural pain in critically ill patients by music therapy: A randomized controlled trial. BMC Complement Med Ther. 2019;26(3):156-165.
  • 6. Candan Dönmez Y, Demir Korkmaz F, Geçit S. Yoğun bakım ünitesindeki çevresel stresörlerin hastalar tarafından algılanması. Türkiye Klinikleri Hemşirelik Bilimleri Dergisi. 2020;12(2):190-197.
  • 7. Çelik S. Yoğun bakım hastalarında ağrı yönetimi. Yoğun Bakım Hemşireliği Dergisi. 2016;20(1):1-8.
  • 8. Topolovec-Vranic J, Canzian S, Innis J, et al. Patient satisfaction and documentation of pain assessments and management after implementing the adult nonverbal pain scale. Am J Crit Care. 2010;19(4):345-354.
  • 9. Yaman Aktaş Y, Karabulut N. The use of cold therapy, music therapy and lidocaine spray for reducing pain and anxiety following chest tube removal. BMC Complement Med Ther. 2019;34:179-184.
  • 10. Herr K, Coyne PJ, Ely E, et al. Pain assessment in the patient unable to self-report: Clinical practice recommendations in support of the ASPMN 2019 position statement. Pain Manag Nurs. 2019;20(5):404-417.
  • 11. Arbour C, Gélinas C. Are vital signs valid indicators for the assessment of pain in postoperative cardiac surgery ICU adults? Intensive Crit Care Nurs. 2010;26(2):83-90.
  • 12. Yaman Aktaş Y, Karabulut N. A Turkish version of the Critical-Care Pain Observation Tool: reliability and validity assessment. J Perianesth Nurs. 2017;32(4):341-351.
  • 13. Gélinas C. Pain assessment in the critically ill adult: Recent evidence and new trends. Intensive Crit Care Nurs. 2016;34:1-11.
  • 14. Herr K, Coyne PJ, McCaffery M, et al. Pain assessment in the patient unable to self-report: Position statement with clinical practice recommendations. Pain Manag Nurs. 2011;12:230-250.
  • 15. Erden S, Demir N, Ugras GA, et al. Vital signs: Valid indicators to assess pain in intensive care unit patients? An observational, descriptive study. Nurs Health Sci. 2018;20(4):502-508.
  • 16. Relland LM, Gehred A, Maitre NL. Behavioral and physiological signs for pain assessment in preterm and term neonates during a nociception-specific response: A systematic review. J Pediatr Neurol. 2019;90:13-23. 17. Hora TCNSD, Alves IGN. Scales for the assessment of pain in the intensive care unit. Systematic review. BrJP. São Paulo, 2020; 3(3):63-74.
  • 18. Durán-Crane A, Laserna A, López-Olivo MA, et al. Clinical practice guidelines and consensus statements about pain management in critically ill end-of-life patients: A systematic review. J Crit Care Med. 2019; 47(11):1619-1626.
  • 19. Al Darwish ZQ, Hamdi R, Fallatah S. Evaluation of pain assessment tools in patients receiving mechanical ventilation. AACN Adv Crit Care. 2016;27(2):162-172.
  • 20. Chanques G, Pohlman A, Kress JP, et al. Psychometric comparison of three behavioural scales for the assessment of pain in critically ill patients unable to self-report. Crit Care. 2014;18(5):1-12.
  • 21. PRISMA 2020 Checklist. Accessed 13 September 2022. https://prisma-statement.org/documents/PRISMA_2020_checklist.pdf
  • 22. Joanna Briggs Institute, JBI Levels of Evidence 2014. Accessed 13 September 2022 https://jbi.global/sites/default/files/2019-05/JBI-Levels-of-evidence_2014_0.pdf
  • 23. Darmanto D, Haloho AB, Zainal R, et al. Suitability of CPOT and BPS to Assess Pain Response in Intubated Mohammad Hoesin Hospital Intensive Care Patients. Journal of Anesthesiology and Clinical Research. 2020;1(2):85-94.
  • 24. Gomarverdi S, Sedighie L, Seifrabiei MA, et al. Comparison of two pain scales: Behavioral pain scale and critical-care pain observation tool during invasive and noninvasive procedures in intensive care unit-admitted patients. Iran J Nurs Midwifery Res. 2019;24(2):151.
  • 25. Ito Y, Teruya K, Nakajima E. Evaluation of pain severity in critically ill patients on mechanical ventilation. Intensive Crit Care Nurs. 2022;68:103118.
  • 26. Juarez P, Bach A, Baker M, et al. Comparison of two pain scales for the assessment of pain in the ventilated adult patient. Dimens Crit Care Nurs. 2010;29(6):307-315.
  • 27. Klein C, Caumo W, Gélinas C, et al. Validation of two pain assessment tools using a standardized nociceptive stimulation in critically ill adults. J Pain Symptom Manage. 2018;56(4):594-601.
  • 28. Kontou P, Kotoulas SC, Kalliontzis S, et al. Evaluation of pain scales and outcome in critically ill patients of a Greek ICU. J Pain Palliat Care Pharmacother. 2022;1-10.
  • 29. Liu Y, Li L, Herr K. Evaluation of two observational pain assessment tools in Chinese critically ill patients. Pain Med. 2015;16(8):1622-1628.
  • 30. Nazari R, Froelicher ES, Nia HS, et al. Diagnostic values of the Critical Care Pain Observation Tool and the Behavioral Pain Scale for pain assessment among unconscious patients: A comparative study. Indian J Crit Care Med. 2022;26(4):472.
  • 31. Rijkenberg S, Stilma W, Endeman H, et al. Pain measurement in mechanically ventilated critically ill patients: Behavioral pain scale versus critical-care pain observation tool. Crit Care. 2015;30(1):167-172.
  • 32. Rijkenberg S, Stilma W, Bosman RJ, et al. Pain measurement in mechanically ventilated patients after cardiac surgery: Comparison of the Behavioral Pain Scale (BPS) and the Critical-Care Pain Observation Tool (CPOT). J Cardiothorac Vasc Anesth. 2017;31(4):1227-1234.
  • 33. Severgnini P, Pelosi P, Contino E, et al. Accuracy of Critical Care Pain Observation Tool and Behavioral Pain Scale to assess pain in critically ill conscious and unconscious patients: Prospective, observational study. J Intensive Care. 2016;4(1):1-8.
  • 34. Wojnar-Gruszka K, Sega A, Płaszewska-Żywko L, et al. Pain assessment with the BPS and CCPOT behavioral pain scales in mechanically ventilated patients requiring analgesia and sedation. International Journal of Environmental Research and Public Health. 2022;19(17):10894.
  • 35. Waladani B, Nazhifah IYESN, Yunita RI. Application of pain scale assessment in patients attached mechanical ventilator in Intensive Care Unit. In 4th International Conference on Sustainable Innovation 2020–Health Science and Nursing (ICoSIHSN 2020) 2021:332-336. Atlantis Press.
  • 36. Bahramnezhad F, Salamat E, Sharifi F, et al. Comparing pain intensity of two instruments in predicting the outcomes of patients under mechanical ventilation admitted to intensive care units. Indian Journal of Pain. 2023;37(1):15-21.
  • 37. Fröhlich MR, Meyer G, Spirig R, et al. Comparison of the Zurich observation pain assessment with the Behavioural Pain Scale and the Critical Care Pain Observation Tool in nonverbal patients in the intensive care unit: A prospective observational study. Intensive and Critical Care Nursing. 2020;60,102874.
  • 38. Gelinas C, Boitor M, Puntillo KA, et al. Behaviors indicative of pain in brain injured adult patients with different levels of consciousness in the intensive care unit. J Pain Symptom Manage. 2019;57(4):761-773.
  • 39. Kotfis K, Zegan-Barańska M, Szydłowski Ł, et al. Methods of pain assessment in adult intensive care unit patients—Polish version of the CPOT (Critical Care Pain Observation Tool) and BPS (Behavioral Pain Scale). Anaesthesiol Intensive Ther. 2017;49(1):66-72.
  • 40. Puntillo K, Gélinas C, Chanques G. Next steps in ICU pain research. Intensive Care Med. 2017;43(9):1386-1388.
  • 41. Vadelka A, Busnelli A, Bonetti L. Comparison between two behavioural scales for the evaluation of pain in critical patients, as related to the state of sedation: An observational study. Scenario. 2017;34(2):4-14.
  • 42. Kawagoe CK, Matuoka JY, Salvetti MG. Instrumentos de avaliação da dor em pacientes críticos com dificuldade de comunicação verbal: revisão de escopo. Revista Dor. 2017;18(2):161-165.
  • 43. Zhai Y, Cai S, Zhang Y. The diagnostic accuracy of critical care pain observation tool (CPOT) in ICU patients: A systematic review and meta-analysis. J Pain Symptom Manage. 2020;60(4):847–856.
  • 44. Wøien H. Movements and trends in intensive care pain treatment and sedation: What matters to the patient? J Clin Nurs. 2020;29(7-8):1129-1140.
  • 45. Faust AC, Rajan P, Sheperd LA, et al. Impact of an analgesia-based sedation protocol on mechanically ventilated patients in a medical intensive care unit. Anesth Analg. 2016;123(4): 903.
  • 46. Bardwell J, Brimmer S, Davis W. Implementing the ABCDE Bundle, Critical-Care Pain Observation Tool, and Richmond Agitation-Sedation Scale to reduce ventilation time. AACN Adv Crit Care. 2020; 31(1):16-21.
There are 45 citations in total.

Details

Primary Language English
Subjects Intensive Care, ICU Nursing
Journal Section Review
Authors

Sevilay Erden 0000-0002-6519-864X

Yeşim Yaman Aktaş 0000-0001-7403-4949

Pınar Kaya 0000-0002-8185-175X

Publication Date April 30, 2025
Submission Date May 2, 2024
Acceptance Date September 6, 2024
Published in Issue Year 2025 Volume: 29 Issue: 1

Cite

 Download Cover Image

Content of this journal is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License

30490