Pengaruh Asisten Virtual Berbasis Artificial Intelligence Terhadap Integritas Sertifikasi Kompetensi Pemrograman secara Online

Imam Prayogo Pujiono; Eko Hari Rachmawanto; Fida Maisa Hana

doi:10.53863/kst.v6i01.1052

Authors

Imam Prayogo Pujiono UIN K.H. Abdurrahman Wahid Pekalongan, Pekalongan, Indonesia
Eko Hari Rachmawanto Universitas Dian Nuswantoro, Semarang, Indonesia
Fida Maisa Hana Universitas Muhammadiyah Kudus, Kudus, Indonesia

DOI:

https://doi.org/10.53863/kst.v6i01.1052

Keywords:

Virtual Assistant; ChatGPT (GPT-4); Certification Integrity; Programming Certification; Online Exam

Abstract

The use of virtual assistants based on Artificial Intelligence (AI) has shown a significant impact in various sectors, especially education. The presence of AI-based virtual assistants capable of answering questions from various topics creates new opportunities and challenges, especially regarding exams and assessments. This research examines the impact of using AI-based virtual assistants, specifically ChatGPT (GPT-4), on the integrity of programming competency certification carried out online. Through field experiments, the author took three online programming competency certifications related to "Python Fundamentals for Beginners", "Java Programming", and "Android Application Development" and used ChatGPT (GPT-4) to answer all exam questions. The test results show that GPT-4 successfully answered 9 out of 10 questions given in the three programming competency certifications that were taken and declared to have passed the exam. This raises serious questions about the validity of certificates obtained through competency certification which is carried out online. This research reveals that there is a need for more effective examination methods in assessing the true abilities of certification participants, such as project-based examinations, examinations using questions in video form, and interview-based examinations. This examination method will not only increase the credibility of online programming competency certification but also ensure that participants who pass the certification have expertise related to the field being tested. Further research is needed to explore the impact of AI-based virtual assistants in various educational contexts, especially on online competency certification.

References

Ahsan, M. M. T., Rahaman, Md. S., & Anjum, N. (2023). From ChatGPT-3 to GPT-4: A Significant Leap in AI-Driven NLP Tools. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.4404397

Arora, S., Athavale, V. A., Himanshu Maggu, & Agarwal, A. (2021). Artificial Intelligence and Virtual Assistant—Working Model (hlm. 163–171). https://doi.org/10.1007/978-981-15-7130-5_12

Bordt, S., & von Luxburg, U. (2023). ChatGPT Participates in a Computer Science Exam. https://doi.org/10.48550/arXiv.2303.09461

de Winter, J. C. F. (2023). Can ChatGPT Pass High School Exams on English Language Comprehension? International Journal of Artificial Intelligence in Education. https://doi.org/10.1007/s40593-023-00372-z

Great Learning. (t.t.). About Us: Great Learning. Diakses pada 20 November 2023 dari https://www.mygreatlearning.com/about-us.

Khairatun Hisan, U., & Miftahul Amri, M. (2023). ChatGPT and Medical Education: A Double-Edged Sword. Journal of Pedagogy and Education Science, 2(01), 71–89. https://doi.org/10.56741/jpes.v2i01.302

King, M. R. (2023). The Future of AI in Medicine: A Perspective from a Chatbot. Annals of Biomedical Engineering, 51(2), 291–295. https://doi.org/10.1007/s10439-022-03121-w

Kirmani, A. R. (2023). Artificial Intelligence-Enabled Science Poetry. ACS Energy Letters, 8(1), 574–576. https://doi.org/10.1021/acsenergylett.2c02758

Leiter, C., Zhang, R., Chen, Y., Belouadi, J., Larionov, D., Fresen, V., & Eger, S. (2023). ChatGPT: A Meta-Analysis after 2.5 Months. https://doi.org/10.48550/arXiv.2302.13795

Lewandowski, M., ?ukowicz, P., ?wietlik, D., & Bara?ska-Rybak, W. (2023). ChatGPT-3.5 and ChatGPT-4 dermatological knowledge level based on the Specialty Certificate Examination in Dermatology. Clinical and Experimental Dermatology. https://doi.org/10.1093/ced/llad255

Mina, P. N. R., Solon, I. M., Sanchez, F. R., Delante, T. K., Villegas, J. K., Basay, F. J., Andales, J., Pasko, F., Estrera, M. F. R., Samson Jr., R., & Mutya, R. (2023). Leveraging Education through Artificial Intelligence Virtual Assistance: A Case Study of Visually Impaired Learners. International Journal of Educational Innovation and Research, 2(1), 10–22. https://doi.org/10.31949/ijeir.v2i1.3001

Nugraheni, B. L. Y., Chrismastuti, A. A., & Sitinjak, E. L. M. (2021). Pedoman Penulisan Karya Ilmiah Dengan Berbagai Paradigma Penelitian. Universitas Katolik Soegijapranata.

Oktavianus, A. J. E., Naibaho, L., & Rantung, D. A. (2023). Pemanfaatan Artificial Intelligence pada Pembelajaran dan Asesmen di Era Digitalisasi. JURNAL KRIDATAMA SAINS DAN TEKNOLOGI, 5(02), 473–486. https://doi.org/10.53863/kst.v5i02.975

OpenAI, :, Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., Avila, R., Babuschkin, I., Balaji, S., Balcom, V., Baltescu, P., Bao, H., Bavarian, M., … Zoph, B. (2023). GPT-4 Technical Report. https://doi.org/10.48550/arXiv.2303.08774

Prawiyogi, A. G., & Toyibah, R. A. (2020). Strategi Peningkatan Kompetensi Mahasiswa Melalui Model Sertifikasi Kompetensi. ADI Bisnis Digital Interdisiplin Jurnal, 1(1), 78–86. https://doi.org/10.34306/abdi.v1i1.103

Rahman, Md. M., & Watanobe, Y. (2023). ChatGPT for Education and Research: Opportunities, Threats, and Strategies. Applied Sciences, 13(9), 5783. https://doi.org/10.3390/app13095783

Rudolph, J., Tan, S., & Tan, S. (2023). ChatGPT: Bullshit spewer or the end of traditional assessments in higher education? Journal of Applied Learning & Teaching, 6(1). https://doi.org/10.37074/jalt.2023.6.1.9

Skalidis, I., Cagnina, A., Luangphiphat, W., Mahendiran, T., Muller, O., Abbe, E., & Fournier, S. (2023). ChatGPT takes on the European Exam in Core Cardiology: an artificial intelligence success story? European Heart Journal - Digital Health, 4(3), 279–281. https://doi.org/10.1093/ehjdh/ztad029

Šlapeta, J. (2023). Are ChatGPT and other pretrained language models good parasitologists? Trends in Parasitology, 39(5), 314–316. https://doi.org/10.1016/j.pt.2023.02.006

Sobania, D., Briesch, M., Hanna, C., & Petke, J. (2023). An Analysis of the Automatic Bug Fixing Performance of ChatGPT. https://doi.org/10.48550/arXiv.2301.08653

Sudrajat, D., Permatasari, R. D., Wijaya, I. M. S., Setyawan, A. E., & Rahayu, N. (2023). Pemanfaatan Kecerdasan Buatan sebagai Upaya Pengembangan Media Pembelajaran Berbasis Multimedia. JURNAL KRIDATAMA SAINS DAN TEKNOLOGI, 5(02), 590–598. https://doi.org/10.53863/kst.v5i02.999

Susnjak, T. (2022). ChatGPT: The End of Online Exam Integrity? https://doi.org/10.48550/arXiv.2212.09292

Takagi, S., Watari, T., Erabi, A., & Sakaguchi, K. (2023). Performance of GPT-3.5 and GPT-4 on the Japanese Medical Licensing Examination: Comparison Study. JMIR Medical Education, 9, e48002. https://doi.org/10.2196/48002

Ventayen, R. J. M. (2023). ChatGPT by OpenAI: Students’ Viewpoint on Cheating using Artificial Intelligence-Based Application. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.4361548