Higher Order Thinking Skills (HOTS) Test Instrument: Validity and Reliability Analysis With The Rasch Model

Rivo Panji Yudha(1*),


(1) Universitas Panca Sakti Bekasi
(*) Corresponding Author

Abstract


This article describes the instruments that can measure students' higher-order thinking skills in learning mathematics for low, medium, and high difficulty levels. This study aims to estimate the validity and determine the reliability of the higher-order thinking test assessment instrument in Mathematics learning based on the Rasch model. The research was conducted through a quantitative descriptive approach in two SMP Negeri 1 Gebang, which had 100 respondents. The research development model used is the ADDIE model. This article describes the stages of development. The instrument used is a high-level mathematical thinking test assessment instrument which contains 20 questions and expert validation observation sheets. Mathematical higher-order thinking test questions were presented to three material experts. The validity used is content validity and construct validity. Reliability was tested through the Rasch model approach. The results showed that the HOTS assessment instrument in the form of HOTS test questions consisting of 20 items of description from the material, construction, and linguistic aspects and the appearance was declared to be constructively valid and suitable for use. The validity results can be seen and analyzed with the Winsteps program in the Out fit order table to see the suitability of items that function in the normal category to measure student misconceptions. The results showed the content validity through the Rasch model approach using Winstep software, where the peak of the graph approached the value 0, the item reliability value was 0.88, the item separation was 1.16, and the response separation was 2.65. The 90% of test questions with moderate criteria

Keywords


Instrument; Assessment; Higher Order Thinking Skills (HOTS); Mathematics; Junior High School

Full Text:

PDF

References


Adayani, S., & Mardapi, D. (2012). Performance Assessment dalam Perspektif Multiple Criteria Decision Making. Prosiding Seminar Nasional Penlitian, Pendidikan Dan Penerapan MIPA.

Aida, N., Kusaeri, K., & Hamdani, S. (2017). Karakteristik Instrumen Penilaian Hasil Belajar Matematika Ranah Kognitif yang Dikembangkan Mengacu pada Model PISA. Suska Journal of Mathematics Education. https://doi.org/10.24014/sjme.v3i2.3897

Ali, R. (2018). Rasch Model Validation Of Instrument To Measure Students Readiness To Embedded Systems Design Course. February.

Arifin, Z., & Retnawati, H. (2017). Pengembangan instrumen pengukur higher order thinking skills matematika siswa SMA kelas X. PYTHAGORAS: Jurnal Pendidikan Matematika. https://doi.org/10.21831/pg.v12i1.14058

Boone, W. J. (2016). Rasch analysis for instrument development: Why,when,and how? CBE Life Sciences Education. https://doi.org/10.1187/cbe.16-04-0148

Boopathiraj, C., & Chellamani, K. (2013). Analysis of test items on difficulty level and discrimination index in the test for research in education. International journal of social science & interdisciplinary research, 2(2), 189-193.

Branch, R. M. (2009). Instructional Design: The ADDIE Approach. In Encyclopedia of Curriculum Studies. https://doi.org/10.4135/9781412958806.n258

Branch, R. M. (2013). Robert Maribe Branch. 24–27. https://doi.org/10.13811/j.cnki.eer.2013.08.005

BSNP. (2010). Paradigma Pendidikan Nasional Abad XXI. In Paradigma Pendidikan Nasional Abad XXI.

Budiman, A., & Jailani, J. (2014). Pengembangan Instrumen Asesmen Higher Order Thinking Skill (Hots) Pada Mata Pelajaran Matematika Smp Kelas Viii Semester 1. Jurnal Riset Pendidikan Matematika. https://doi.org/10.21831/jrpm.v1i2.2671

Chan, S. W., Looi, C. K., & Sumintono, B. (2020). Assessing computational Thinking Abilities Among Singapore Secondary Students: a Rasch Model Measurement Analysis. Journal of Computers in Education, 0123456789. https://doi.org/10.1007/s40692-020-00177-2

Christensen, K. B., Makransky, G., & Horton, M. (2017). Critical Values for Yen’s Q3: Identification of Local Dependence in the Rasch Model Using Residual Correlations. Applied Psychological Measurement. https://doi.org/10.1177/0146621616677520

Conrad, K. J., Conrad, K. M., Mazza, J., Riley, B. B., Funk, R., Stein, M. A., & Dennis, M. L. (2017). Dimensionality, Hierarchical Structure, Age Generalizability, and Criterion Validity of the GAIN’s Behavioral Complexity Scale. Physiology & Behavior, 176(1), 139–148. https://doi.org/10.1016/j.physbeh.2017.03.040

Deane, T., Nomme, K., Jeffery, E., Pollock, C., & Birol, G. (2016). Development of the statistical reasoning in biology concept inventory (SRBCI). CBE Life Sciences Education. https://doi.org/10.1187/cbe.15-06-0131

Etikan, I., Musa, S. A., & Alkassim, R. S. (2016). Comparison of convenience sampling and purposive sampling. American journal of theoretical and applied statistics, 5(1), 1-4.

Gustafson, K. L., & Branch, R. M. (2002). What is Instructional Design. Trends and Issues in Instructional Design and Technology, 2, 10–16.

Hess, A. K. N., & Greer, K. (2016). Designing For Engagement: Using The ADDIE Model To Integrate High-Impact Practices Into An Online Information Literacy Course. Communications in Information Literacy, 10(2), 264–282.

Janssen, K. C., Phillipson, S., O’Connor, J., & Johns, M. W. (2017). Validation of the Epworth Sleepiness Scale for Children and Adolescents using Rasch analysis. Sleep Medicine, 33, 30–35. https://doi.org/10.1016/j.sleep.2017.01.014

Linacre, J. M. (1994). Sample Size and Item Calibration or Person Measure Stability. Rasch Measurement Transactions, 7(4), 328.

Mahmud, Z., & Porter, A. (2015). Using rasch analysis to explore what students learn about probability concepts. Journal on Mathematics Education. https://doi.org/10.22342/jme.6.1.1937.1-10

Muruganantham, G. (2015). Developing of E-Content Package By Using ADDIE Model. 1, 52–54.

Nisrokha. (2020). Tes Terstandarisasi. Jurnal Madaniyah, 10(1), 15–36.

Pratama, G. S., & Retnawati, H. (2018). Urgency of Higher Order Thinking Skills (HOTS) Content Analysis in Mathematics Textbook. Journal of Physics: Conference Series. https://doi.org/10.1088/1742-6596/1097/1/012147

Purbaningrum, K. A. (2017). Kemampuan Berpikir Tingkat Tinggi Siswa Smp Dalam Pemecahan Masalah Matematika Ditinjau Dari Gaya Belajar. Jurnal Penelitian Dan Pembelajaran Matematika. https://doi.org/10.30870/jppm.v10i2.2029

Rahmah, A. N., & Muharni, L. P. J. (2019). Identifikasi Soal Tipe Higher Order Thinking Skills (Hots) Pada Buku Matematika Materi Persamaan Dan Pertidaksamaan Linear Satu Variabel. Edu Math Journal Prodi Pendidikan Matematika.

Rofiqoh, A. (2014). Analisa Soal-Soal Pada Buku Siswa Pelajaran Matematika Smp Kelas Vii Kurikulum 2013 Berdasarkan Taksonomi Bloom Terevisi. Jurnal Pembelajaran Dan Pengembangan Matematika (PEMANTIK). https://e-journal.unmas.ac.id/index.php/pemantik/article/view/1350

Rufiana, I. S. (2016). Level kognitif soal pada buku teks matematika kurikulum 2013 kelas VII untuk pendidikan menengah. Jurnal Dimensi Pendidikan dan Pembelajaran, 2(2), 13-22. https://doi.org/10.24269/dpp.v2i2.153

Runnels, J. (2012). Using the Rasch model to validate a multiple choice English achievement test. International Journal of Language Studies.

Said, R. F. M. (2016). Application of Rasch Measurement Model in Evaluating Student Performance for Foundation of Computing II. In 7th International Conference on University Learning and Teaching (InCULT 2014) Proceedings (Issue InCULT). https://doi.org/10.1007/978-981-287-664-5_58

Sarjono. (2015). Penilaian Unjuk Kerja Dalam Praktikum Fisika. Jurnal Madaniyah, 5(1), 109–130. https://journal.stitpemalang.ac.id/index.php/madaniyah/article/view/69

Schraw Ed., G., & Robinson Ed., D. H. (2011). Assessment of Higher Order Thinking Skills. Current Perspectives on Cognition, Learning and Instruction. In IAP - Information Age Publishing, Inc.

Sharifah, R., & Faaizah, S. (2015). The Development of Online Project Based Collaborative Learning using ADDIE Model. Procedia - Social and Behavioral Sciences, 195, 1803–1812. https://doi.org/10.1016/j.sbspro.2015.06.392

Sumintono, B. (2018, February). Rasch model measurements as tools in assesment for learning. In 1st International Conference on Education Innovation (ICEI 2017) (pp. 38-42). Atlantis Press.

Sumintono, B., & Widhiarso, W. (2015). Aplikasi pemodelan rasch pada assessment pendidikan. Trim komunikata.

Sumintoro, Bambang, Widhiarso, W. (2013). Model Rasch Untuk Penelitian Ilmu-Ilmu Sosial (B. Sumintoro (ed.); II).

Suryapuspitarini, B. K., Wardono, & Kartono. (2018). Analisis Soal-Soal Matematika Tipe Higher Order Thinking Skill ( HOTS ) pada Kurikulum 2013 untuk Mendukung Kemampuan Literasi Siswa. Prisma, Prosiding Seminar Nasional Matematika.

Tabatabaee-Yazdi, M. et. al. (2018). Development and Validation of a Teacher Success Questionnaire Using the Rasch Model. International Journal of Instruction, 11(2), 129–144. https://doi.org/10.12973/iji.2018.11210a

Torrey, T., & Emrah, P. (2019). Using the ADDIE Model and Universal Design for Learning Principles to Develop an Open Online Course for Teacher Professional Development. Journal of Digital Learning in Teacher Education, 34(4), 219–233. https://doi.org/10.1080/21532974.2018.1494521

Wardhani, D. F., & Putra, A. P. (2016). Pengembangan Instrumen Tes Standar Kognitif pada Mata Pelajaran IPA Kelas 7 SMP Di Kabupaten Banjar The Development of Standardized Test Instrument Natural Science Subjects Seventh Grade Junior High School in Kabupaten Banjar. Proceeding Biology Education Conference, 13(1), 75–82.

Wirda, M. A., Berutu, N., Rahmad, R., & Rohani, R. (2017). Pengembangan Tes Standar Berasis Teknologi Informasi Di Jurusan Pendidikan Geografi. Tunas Geografi. https://doi.org/10.24114/tgeo.v6i2.8548

Yudha, R. P. (2016). Pengembangan Instrumen Unjuk Kerja Materi Bangun Ruang Di Sekolah Dasar Kota Cirebon. Eduma : Mathematics Education Learning and Teaching. https://doi.org/10.24235/eduma.v5i2.1079

Yudha, R. P. (2020). Validity And Reliability Rubric Of Performance Assessment Geometry Study In Junior High School Using The Many Facet Rasch Model Approach. Eduma : Mathematics Education Learning and Teaching. https://doi.org/10.24235/eduma.v9i2.7100

Zulyadaini, D. (2017). A Development of Students’ Worksheet Based on Contextual Teaching and Learning. IOSR Journal of Mathematics, 13(01), 30–38. https://doi.org/10.9790/5728-1301033038




DOI: 10.24235/eduma.v12i1.9468

Article Metrics

Abstract view : 0 times
PDF - 0 times

Refbacks

  • There are currently no refbacks.


Bookmark and Share


Copyright (c) 2023

View My Stats