Date of Award

Fall 8-17-2020

Degree Type

Dissertation

Degree Name

EdD Education Leadership, Management and Policy

Department

Education Leadership, Management and Policy

Advisor

Christopher H. Tienken, Ed.D.

Committee Member

Samuel F. Fancera, Ed.D.

Committee Member

Paige D. Sydoruk, Ed.D.

Keywords

cognitive complexity, creative thinking, higher-order thinking, problem solving, standardized assessment, strategic thinking, Webb’s Depth of Knowledge (DOK)

Abstract

ABSTRACT

The importance of higher-order thinking, 21st-Century Skills, and standardized testing are important issues in education. TerraNova and IOWA tests are widely used to assess students’ academic competence. The publishers of the TerraNova and IOWA tests claim that their assessment instruments challenge students to employ higher-order thinking skills. This study sought to examine and describe ways in which the language found in the TerraNova and IOWA English/Language Arts practice tests compared with the language that promotes higher-order thinking found in the literature. A convergent, parallel mixed-methods study with qualitative and quantitative content analysis methods was conducted to (a) describe the way(s) in which the language found in the questions on the TerraNova and IOWA English/Language Arts Practice Tests for grade 8 compares with the language associated with higher-order thinking found in research literature and (b) to describe and compare the complex thinking requirements found on the TerraNova and the IOWA English/Language Arts Practice Tests for grade 8. The qualitative method consisted of a content analysis of the language of the questions on each test, deductive coding, and categorizing the cognitive level of each question based on Webb’s Depth of Knowledge. The quantitative aspect of the study consisted of calculating the percentages of questions categorized in each level of Webb’s Depth of Knowledge. Each depth of knowledge level represents a level of cognitive complexity: levels 1 and 2 – lower level, levels 3 and 4 – higher level. Each question was rated on a 1–4 DOK level based on Webb’s Depth of Knowledge methodology. To assist with reliability in coding, the coders utilized a double-rater read behind consensus model. This study sought to reveal the extent to which complex/higher-order thinking skills are incorporated throughout the TerraNova and IOWA standardized practice tests. The results of this convergent, parallel mixed-methods study found that 98% of the 220 test questions analyzed are promoting lower level thinking: Level 1 questions accounting for 76%, and Level 2 questions accounting for 22%. Of the questions analyzed and coded, only 3 questions were categorized as DOK Level 3, equating to 2% of all questions analyzed and scored in this study. No questions analyzed in this study were categorized as DOK Level 4, equating to 0% of all questions reviewed. The results suggest that the questions promote functional fixedness as opposed to higher-order thinking. Questions categorized as levels 1 and 2 of Webb’s DOK are lower level questions with a focus on declarative and procedural knowledge (recall, reconstruction). A consistent focus on lower level declarative and procedural thinking can stunt the complex thinking development of students and lead to functional fixedness. Functional fixedness is known as the phenomenon of one perceiving an entity as having only one function or use. It is a mental set that incites myopic thinking so that an individual is unable see alternatives.

Keywords/terms: cognitive complexity, creative thinking, higher-order thinking, problem solving, standardized assessment, strategic thinking, Webb’s Depth of Knowledge (DOK)

Download

Share

COinS