Journal of Research in Science, Mathematics and Technology Education

The Development and Validation of a 21st Century Skills Instrument: Measuring Secondary School Students’ Skills

Alpaslan Sahin 1 * ,
Myeongsun Yoon 2,
Mirim Kim 1
1 Harmony Public Schools2 Texas A&M University* Corresponding Author
Download PDF
Journal of Research in Science, Mathematics and Technology Education, Volume 2, Issue 2, May 2019, pp. 85-103
OPEN ACCESS VIEWS: 800 DOWNLOADS: 1834 Publication date: 15 May 2019
ABSTRACT
Due to the rapid change in technology and information dissemination, the qualities and skills employers  and colleges demand in the 21st century have changed. To help higher education institutions and workforce to identify and measure their prospective students and employees’ skills respectively, we designed an instrument for  secondary grade students to self-assess their 21st century skills. After successful piloting, validation of the final  instrument was done with 282 high school students from a public high school in Texas. We utilized exploratory factor analysis and investigated construct validity for the instrument using principal axis factoring with Promax  rotation and Kaiser normalization. We found that the original 48 items developed for the instrument were loading the  four factors as theorized in our model. Finally, confirmatory factor analysis (CFA) models for four scales were separately investigated. Maximum likelihood estimation method was used for all anal yses though Mplus8.2 (Muthén & Muthén, 1998-2017). We came up with 5 factors and 43 items. Researchers, K-12 educators, postsecondary  educators, and employers may benefit from the development of this instrument.   
KEYWORDS
21st century skills, Exploratory factor analysis, Instrument development, Principal axis factoring, Confirmatory factor analysis.
CITATION (APA)
Sahin, A., Yoon, M., & Kim, M. (2019). The Development and Validation of a 21st Century Skills Instrument: Measuring Secondary School Students’ Skills. Journal of Research in Science, Mathematics and Technology Education, 2(2), 85-103. https://doi.org/10.31756/jrsmte.223
REFERENCES
  1. Anderson, L. W., Krathwohl, D. R., Airasian, P., Cruikshank, K., Mayer, R., Pintrich, P., ... Wittrock, M. (2001). A taxonomy for learning, teaching and assessing: A revision of
  2. Bloom’s taxonomy. New York. Longman Publishing.
  3. Association of American Colleges and Universities. (2007). College learning for the new global century: A report from the National Leadership Council for Liberal Education & America's Promise. Washington, DC: Author. Retrieved from
  4. https://www.aacu.org/sites/default/files/files/LEAP/GlobalCentury_final.pdf
  5. Bell, S. (2010). Project-based learning for the 21st century: Skills for the future. The Clearing House, 83(2), 39–43.
  6. Binkley, M., Erstad, O., Herman, J., Raizen, S., Ripley, M., Miller-Ricci, M., & Rumble, M. (2012). Defining twenty-first century skills. In P. Griffin, B. McGaw, & E. Care (Eds.), Assessment and teaching of 21st century skills (pp. 17–66). New York: Springer.
  7. Bloom, B. S. (Ed.) (1984). Taxonomy of educational objectives (Handbook 1: Cognitive domain). New York: Longman.
  8. Brown, J. D. (2009). Choosing the right type of rotation in PCA and EFA. JALT Testing and Evaluation SIG Newsletter, 13(3), 20–25.
  9. Calvani, A., Cartelli, A., Fini, A., & Ranieri, M. (2009). Models and instruments for assessing digital competence at school. Journal of E-learning and Knowledge Society, 4(3), 183– 193.
  10. Churches, A. (2008). Bloom’s digital taxonomy. Retrieved from http://burtonslifelearning.pbworks.com/f/BloomDigitalTaxonomy2001.pdf
  11. Dede, C. (2010). Comparing frameworks for 21st century skills. In J. A. Bellanca & R. S. Brandt (Eds.), 21st century skills: Rethinking how students learn (pp. 51–76). Bloomington, IN:
  12. Solution Tree Press.
  13. Dron, J., & Anderson, T. (2014). Teaching crowds: Learning and social media. Edmonton, Canada: Athabasca University Press.
  14. Fan, S., & Le, Q. (2011). Developing a valid and reliable instrument to evaluate users’ perception of web-based learning in an Australian university contexts. Journal of Online Learning and Teaching, 7(3), 366–379.
  15. Greiff, S., & Kyllonen, P.C. (2016). Contemporary assessment challenges: The measurement of 21st century skills. Applied Measurement in Education, 29(4), 243–244.
  16. Griffin, P., & Care, E. (Eds.). (2015). Assessment and teaching of 21st century skills: Methods and approach. New York: Springer.
  17. International Society for Technology in Education. (2016). ISTE standards for students. Retrieved from http://www.iste.org/
  18. Lawshe, C. H. (1975). A quantitative approach to content validity 1. Personnel Psychology, 28(4), 563–575.
  19. Levy, F., & Murnane, R. J. (2004). The new division of labor: How computers are changing the way we work. Princeton, NJ: Princeton University Press.
  20. Lombardi, M. M. (2007). Authentic learning for the 21st century: An overview. In D. G.
  21. Oblinger (Ed.), Educause Learning Initiative: Advancing learning through IT innovation (pp. 1–12). Retrieved from https://library.educause.edu/resources/2007/1/authenticlearning-for-the-21st-century-an-overview
  22. Mintz, S. (2014, March 5). Five ways that 21st and 20th century learning will differ: Proficiency, data, science, and more. Inside Higher Ed. Retrieved from https://www.insidehighered.com/blogs/higher-ed-gamma/five-ways-21st-and-20thcentury-learning-will-differ
  23. Mishra, P., & Kereluik, K. (2011). What 21st century learning? A review and a synthesis. In M.
  24. Koehler & P. Mishra (Eds.), Proceedings for the 22nd International Conference for the Society for Information Technology & Teacher Education (pp. 3301–3312). Chesapeake, VA: Association for the Advancement of Computing in Education.
  25. National Center on Education and the Economy. (2007). Tough choices or tough times: The report of the new commission on the skills of the American workforce. Washington, DC: Author. Retrieved from http://www.ncee.org/wp-content/uploads/2010/04/ExecutiveSummary.pdf
  26. North Central Regional Educational Laboratory, & Meriti Group. (2003). enGauge® 21st Century Skills: Literacy in the Digital Age.
  27. Organisation for Economic Cooperation and Development. (2005). The definition and selection of key competencies: Executive summary. Paris, France: OECD.
  28. Osman, K., Soh, T. M. T., & Arsad, N. M. (2010). Development and validation of the Malaysian 21st century skills instrument (M-21CSI) for science students. Procedia-Social and Behavioral Sciences, 9, 599–603.
  29. Pacific Policy Research Center. (2010). 21st century skills for students and teachers. Honolulu: Kamehameha Schools. Retrieved from
  30. http://www.ksbe.edu/_assets/spi/pdfs/21_century_skills_full.pdf
  31. Partnership for 21st Century Learning. (2009). Results that matter: 21st Century skills and high school reform. Tucson, AZ: Author.
  32. Partnership for 21st Century Learning. (2016). Framework for 21st century learning. Washington, DC: Author. Retrieved from www.p21.org
  33. Piaget, J. (1950). The psychology of intelligence. Routledge.
  34. Rich, E. (2010). How do you define 21st-century skills? One question. Eleven answers.
  35. Education Week, 4(1), 32–35.
  36. Rotherham, A. J., & Willingham, D. T. (2010). “21st-Century” Skills. American Educator, 17.
  37. | S A H I N , K I M , & Y O O N
  38. Saavedra, A. R., & Opfer, V. D. (2012). Learning 21st-century skills requires 21st-century teaching. Phi Delta Kappan, 94(2), 8–13.
  39. Saettler, P. (2004). The evolution of American educational technology. Greenwich, CT:
  40. Information Age Publishing.
  41. Schmidt, D. A., Baran, E., Thompson, A. D., Mishra, P., Koehler, M. J., & Shin, T. S. (2009). Technological pedagogical content knowledge (TPACK) the development and validation of an assessment instrument for preservice teachers. Journal of Research on Technology in Education, 42(2), 123–149.
  42. Schneider, S. (1997). Defining environmental literacy. Trends in Ecology & Evolution, 12(11), 457.
  43. Silva, E. (2008). Measuring skills for the 21st century (Education Sector Reports). Washington, DC: Education Sector.
  44. Silva, E. (2009). Measuring skills for 21st-century learning. Phi Delta Kappan, 90(9), 630–634.
  45. Tabachnick, B. G. & Fidell, L. S. (2007). Using multivariate statistics (5th ed.). Upper Saddle River, NJ: Pearson Allyn & Bacon.
  46. Trilling, B. & Fadel, C. (2009). 21st century skills: Learning for life in our times. Retrieved from http://21stcenturyskillsbook.com/blog/q-a/
  47. Voogt, J., & Roblin, N. P. (2012). A comparative analysis of international frameworks for 21st century competences: Implications for national curriculum policies. Journal of Curriculum Studies, 44(3), 299–321.
LICENSE
Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.