Research Reports & Publications
To view Kingsbury Center presentations, please visit our Slideshare page.
| Report Name |
Published |
Author(s) |
|---|---|---|
Clemson University Researchers Evaluating Professional Development Program Using NWEA DataJeff Marshall and Bob Horton from Clemson University’s Inquiry in Motion Institute are working with the Kingsbury Center on the evaluation component of a multi-year study examining the student learning impact of a Professional Development (PD) program for math and science teachers. Learn More About the Study |
September 7, 2010 | Rebecca Moore |
School Conditional Growth ModelAs more schools are held accountable for current state educational accountability systems, many criticisms have been leveled at the inadequacy and inconsistency of status-based measures. This study helps to view school effectiveness as a measurement of growth instead of simple decisions regarding passing or failing at one-time point or even multi-time points. To further address the assertion that schools with a large proportion of academically disadvantaged students are usually evaluated and judged unfairly, this study takes into account school context and develops conditional school growth to make the “apples to apples” comparison possible. |
June 28, 2010 | |
Virtual Comparison Groups: Impact of Students with Insufficient Matches on Educational EvaluationsRandomized controlled experiments are difficult to implement in a school setting. While it is feasible to randomly assign patients to treatment conditions in some medical trials, this same practice can frequently be problematic when applied in school settings. Virtual Comparison Groups (VCG) were developed as an alternative quasi-experimental design for social science researchers working in the field of education. Learn More About the Study |
May 5, 2010 | |
Evaluating the Effect of Random Selection on Virtual Comparison Group CreationVirtual Comparison Groups (VCG) were developed by the Northwest Evaluation Association as an alternative to conventional controlled experiments for social science researchers working in the field of education. This study evaluated the procedures for selecting students to be used as Virtual Comparisons for a study group of 1,000 students in an effort to determine whether the random selection procedures created differences between the VCG sample and the population from which it was drawn. |
May 5, 2010 | |
An Investigation of the Relationship between Time of Testing and Test-Taking EffortThis study investigated the relationships between when a test is administered and the amount of test-taking effort exhibited by examinees. Three time-related variables were investigated: the time of year the test was administered, the day of the week the test event occurred, and the time of day that the test event occurred. Mean effort did not appear to vary across either time of year nor day of week. There was, however, a clear time of day effect, with mean effort decreasing throughout the day. In addition, mean effort was found to decreases across grade, and male examinees were found to exhibit lower mean effort than female examinees. |
April 28, 2010 | |
State Standards and Student Growth: Why State Standards Don’t Matter as Much as We ThoughtThis is a study of the relationship between student growth and state standards, specifically examining two questions: 1) Does the difficulty of a state’s proficiency standards bear any relationship to student academic growth? 2) Do students who are above their state’s proficiency standards demonstrate less growth, relative to their peers, than students performing below the level of their state proficiency standards? Findings include that what did impact growth was not whether state proficiency standards were high or low, but whether a student was above or below that arbitrary proficiency line, wherever it was. Students above that line received less benefit from their instruction, relative to peers, than did students below that line. This finding has implications for higher performing students. |
February 1, 2010 | |
Achievement Gaps and the Proficiency TrapA new Kingsbury Center analysis of the way achievement gaps are calculated reveals the potential for distorted perception of how different groups of students are really doing. The report “Achievement Gaps and the Proficiency Trap” uses data from NWEA’s MAP assessments to look at achievement gaps in different states, illustrating the variety of ways the data could be interpreted by using different measures or different state standards. The report also suggests new ways to look at data that will provide a better measure of the difference in performance between two types of students. View the Data Gallery |
January 21, 2010 | |
Individual Score Validity in a Modest-Stakes Adaptive Educational Testing SettingThe purpose of this study is to examine the immediate consequences of ignoring odd or unexpected test-taker interactions that could adversely affect proficiency estimates in operational, low- to moderate-stakes computerized adaptive tests. Immediate consequences, here, are considered to be the effects on resulting proficiency estimates and their standard errors. The study focuses on test events that are “suspicious” from a test-test taker interaction perspective. This paper was presented at the 2009 AERA national conference. |
April 16, 2009 | |
The Accountability IllusionThis study, conducted in partnership with the Thomas B. Fordham Institute, examines the No Child Left Behind system and Adequate Yearly Progress (AYP) rules for 28 states. Thirty-six schools were studied to determine which of them would or would not make AYP when evaluated under each state's accountability rules. Based on this analysis, we can see how AYP varies across the country and evaluate the effectiveness of NCLB. |
February 19, 2009 | |
The Proficiency IllusionThis study investigated expectations around the No Child Left Behind Act (NCLB) policy that requires all students to be “proficient” in reading and math by 2014. This study used data from schools whose pupils participated both in state testing and in assessment by the Northwest Evaluation Association (NWEA) to estimate proficiency cutscores (the level students need to reach in order to pass the test for NCLB purposes) for assessments in twenty-six states. The authors compared states’ definitions for proficiency, changes in states’ definitions for proficiency, and how proficiency standards are calibrated across grades. |
October 4, 2007 | |
Achievement Gaps: An Examination of Differences in Student Achievement and GrowthThe difference between the academic performance of poor students and wealthier students and between minority students and their non-minority peers is commonly known as the achievement gap. This study examines the achievement gap using a large sample of students from a wide variety of school districts across the United States. The studies in this report focus on the growth of individual students and have taken the analysis even further by looking at individual student growth across more than just two points in time. They also focus more precisely on the growth of students with a particular starting point. |
November 13, 2006 | |
The Impact of the No Child Left Behind Act on Student Achievement and Growth: 2005 EditionThis study compares student achievement and student growth in achievement prior to the implementation of NCLB and the year following implementation. It also examines the impact of NCLB on the performance and growth of students in several ethnic groups to investigate the promise of equity implicit in NCLB. |
April 8, 2005 | |
Computerized adaptive testing and No Child Left BehindThis study investigates several testing approaches and their potential impact on the use of test scores for these purposes such as identifying whether individual students are proficient, to helping determine whether schools are causing adequate growth for their students. |
April 13, 2004 | |
Individual Growth and School SuccessThe primary point of this study is to investigate whether, and to what extent, current Adequate Yearly Progress (AYP) definitions could be improved by adding an individual growth measure to reflect the percentage of students who are currently identified as proficient. |
April 1, 2004 | |
The State of State StandardsIn this study, NWEA compiled a meta-analysis of 14 prior research studies investigating the student proficiency standards that have been established by various states. |
December 21, 2003 | |
How strong is the incentive for educators to game the Adequate Yearly Progress requirements of the No Child Left Behind Act?The purpose of this paper is to raise the concern that educators may adopt practices (in their efforts to comply with NCLB), that are counterproductive to the law’s intent, and to provide evidence supporting this concern. |
September 1, 2003 | John Cronin |
Pursuing reasonable individual student growth targetsThe purpose of this study is to evaluate several feasible models for determining single-year academic growth targets for individual students. |
April 1, 2003 | Carl Hauser |