Date of Award

Spring 5-19-2018

Degree Type


Degree Name

EdD Education Leadership, Management and Policy


Education Leadership, Management and Policy


Christopher Tienken, Ed.D.

Committee Member

Anthony Colella, Ph.D.

Committee Member

Dario Sforza, Ed.D.


higher order thinking, educational technology, 21st century skills, cognitive complexity, critical thinking


Higher order thinking skills have been embedded into state teaching standards, including Standard 9 of the New Jersey Core Curriculum Content Standards (NJCCCS) for 21st Century Life and Careers. These standards aim to encourage problem solving, critical thinking, reasoning, and real-world application in order to prepare students for the workplace and are expected to be embedded consistently in classroom practice and learning. With the promotion of Standard 9 in New Jersey schools, technology has become a paramount tool because of its multitude of capabilities. Over the years, private companies have begun developing online-based programs catered to building content area skills as well as higher order thinking skills.

Although these online-based programs have gained popularity in recent years, there is little research conducted on the effectiveness of these programs, the validity of the claims made by these private companies, or the types and frequency of tasks that promote higher order thinking skill set development embedded in such programs. The objective of this study was to describe the level and distribution of cognitive complexity within an online-based program using a qualitative content analysis. This program, given the pseudonym of HOT Learning, markets itself as promoting higher order thinking skillset development among students Grades K–8. Two coders utilized Hess’ Cognitive Rigor Matrix to examine 231 questions from the HOT Learning program following the double-rater read-behind consensus model. These questions aligned with NJCCCS for Grade 8 English language arts. The findings in this study showed that the majority of questions did not place into higher order thinking cells within Hess’ Cognitive Rigor Matrix.

This study serves as an examination of online-based programs that market the development of higher order thinking skills, raising a need for critique of these programs from school administrators and stakeholders before the purchase and implementation of such programs. There is a need for school personnel to examine these programs in order to decide on one that best meets students’ needs and the intended goal of how the program would be used. It is suggested that adequate supports in the form of professional development and training be put in place in order to facilitate teacher use of these programs. Furthermore, there is a demand for the New Jersey Department of Education (NJDOE) to assess and monitor the validity of these programs compared to the marketing and provide state-approved professional development to teachers not only in the use of these programs but in the understanding and practice of encouraging higher order thinking skills.