Date of Award

Fall 10-21-2021

Degree Type

Dissertation

Degree Name

EdD Education Leadership, Management and Policy

Department

Education Leadership, Management and Policy

Advisor

Eloise Marks-Stewart, Ed.D.

Committee Member

David Reid, Ph.D.

Committee Member

Michael Thumm, Ed.D.

Keywords

Hess’ Cognitive Rigor Matrix, HCRM, Cognitive Rigor, higher-order thinking skills, college and career readiness, high stakes standardized testing, PSAT, PSAT/NMSQT

Abstract

As educators and school leaders work towards building students’ capacity throughout a student's academic career, the goal is to facilitate opportunities for students to develop college and career readiness skills that can be applied to novel and challenging problems long after high school graduation. Hess’ Cognitive Rigor Matrix (HCRM) is a tool that objectively and effectively measures prompts, learning exercises, or test questions by blending the Revised Bloom’s Taxonomy and Webb’s Depth of Knowledge. Thus, a learning environment that cultivates higher-order thinking skills can be accurately examined and provide educators, researchers, and school leaders with data to make more informed decisions about delivering instruction and proficiently assessing the students' college and career readiness. Increasingly, the culture of education in the United States has been impacted by the results of high-stakes standardized testing. Essentially, the significance of the Preliminary Scholastic Aptitude Test and National Merit Scholarship Qualifying Test (PSAT/NMSQT) has grown to influence a student’s academic trajectory, a high school’s evaluation. The PSAT/NMSQT has also contributed to a standardized testing industry and test-prep industry that has blossomed into the billion-dollar range. Moreover, the PSAT/NMSQT purportedly measures a student’s college and career readiness. Given the vast scope and depth of consequences that accompany students’ PSAT/NMSQT scores, the exam’s claims to examine career and college readiness skills were thoroughly investigated through the lens of qualitative document analysis. The study was able to identify and quantify the higher-order thinking skills of the 2018 PSAT/NMSQT assessed on the examination using subject-specific HCRMs.

This research has added an objective measure of the language used on the 2018 PSAT/NMSQT and found that the extent to which higher-order thinking is being used on the exam does not support its stated purpose of assessing college readiness. The researchers methodically and systematically reviewed the language of each question on the 2018 PSAT/NMSQT through the lens of subject-specific HCRMs to determine the overall level of cognitive rigor of the 2018 PSAT/NMSQT and the level of cognitive rigor of each section (Reading, Writing, and Math). Overall, 31.6% of the questions in the 2018 PSAT/NMSQT contained language compared to the specified HCRMs definition of higher-order thinking. Significantly, most questions (51.8%) on the 2018 PSAT/NMSQT required test-takers to simply apply skills and concepts to solve the test question. The study concluded that the PSAT/NMSQT should look to enhance the math section in order to improve the cognitive rigor required to complete the test question. Additionally, local education agencies should create more effective learning experiences for their students to practice higher-order thinking skills rather than relying on a high-stakes standardized test. Lastly, Hess’ Cognitive Rigor Matrix is a well-acquitted, fully researched measurement tool that can be applied to local assessments and develop a culture of deeper learning through a validated, yet differentiated catalog of data-driven assessments that inspires students to engage in challenging learning experiences.

Download

Share

COinS