Data Award Q & A

1.      Do you have start and end dates for the school year?

o   We do not systematically collect start and end dates for the school year. 

2.      Is it reasonable to compare growth across groups that start at different places on the vertically-aligned scale?

o   No.  In our own work, we always adjust growth comparisons to account for differences in starting position on the scale.  Thus the growth norms we use with schools show growth for students not only at each grade, but also at each starting score within a grade.  When we employ HLM methodologies, we include starting score in the model.  And when we create Virtual Comparison Groups, each member of a control group starts with the same score as a study group member.

3.      Are all students tested twice yearly, so that we can compare school-year and summer learning?

o   All MAP licenses allow for testing up to 4 times per year.  Most schools test at least twice per year, though a small percentage only test once.  In regard to comparisons around summer learning, that is an issue that is of particular interest to us.  And we have good data to explore it.

4.      What types of projects have been awarded in the recent past? May I access those?

o   We do not have any finalized studies from the Data Award program to share at this time, but some examples of past awarded projects include analyses of student mobility patterns and the academic growth of deaf and hard of hearing students.

5.      What types of research questions is NWEA interested in?

o   Our website can give you some good examples of the types of research we engage in.  Please go to About Our Research to learn more about the areas of education research we undertake.  Also be sure to check out our reports page.

6.      Do you have reports available on summer learning?

o   No.  That’s one reason why are particularly interested in well-designed studies in this area.

7.      If we are interested in studying particular programs that would require data to be de-identified at the school/district level, will that be possible?

o   Our assumption is that the school or district might be identified but not individual students.  If our assumption is correct, if the researcher agrees NOT to identify the schools and districts in any published report stemming from the study.  If schools and/or districts will (or can) be identified in reports stemming for the study, then we must have written permission in the form of a data release, from all schools that would be included.  We would work with the researcher to determine how to best obtain that release, but it is a step you will want to consider when you are thinking about the scope of work for your project. 

8.      Are teacher growth summaries available for particular types of teachers (minus the teacher's name of course)?

o   No.

9.      We may be interested in understanding socioeconomic status beyond race and ethnicity. Can socioeconomic data be linked in from NCES? Are NCES data available for all schools, or only for a subset?

o   NCES data are available for all schools who participate in the survey.  We access the publicly available dataset (Common Core of Data) and link it into the GRD. One of the CCD fields is the number and percentage of Free and Reduced Lunch students at a school.

10.  If I have a question that is very specific to my topic, who may run it by first before applying?

o   Please submit additional questions to kingsburycenter@nwea.org .

11.  Is there any way to link to neighborhood information, such as average household income or home price?

o   At this time we are not systematically linked into census data, though we could feasibly look up individual school addresses and make the links that way.  We would still need to maintain our school and district privacy policy.

12.  How does data access work? Do researchers log in to your data warehouse, or do you extra data and send it to winning researchers?

o   Researchers will be provided with their dataset (created per the specifications that are developed with Kingsbury Center staff) via our secure ftp website.

13.  Is this slideshow available to us afterwards for a reference?

o   Yes – you should be able to access it on the Adobe Connect website through your meeting invite link.  We also expect to make the presentation available on the kingsburycenter.org website.

14.  Are schools distinguished by sector: neighborhood public schools vs private, charter, magnet?

o   Yes, we can code and aggregate school type using the NCES data.

15.  Do the reading/math scales guard against ceiling effects?

o   Yes (but all tests ultimately have some ceiling).  Because the reading and mathematics tests are adaptive, they have much lower standard errors of measure at the upper end of the achievement distribution then fixed form assessments.  Over the past several years, we have also significantly increased the number of items at the upper end of the measurement scale which has helped reduce ceiling effect artifacts.  In reading, the test measures quite accurately through a college entrance level of achievement.  Students in the upper five percent of our norms in ninth and ten percent of our norms in tenth grades might be underestimated some because of a ceiling.  In mathematics, high performing students generally move from a general math assessment to end-of-course assessments in mathematics (Algebra I, Algebra II, Geometry, Integrated Math 1 and 2), usually in eighth grade.  End-of-course tests are reported on the same scale as the general math assessment and have considerably more range.  We would work with the researcher on ways to interpret data across these tests.

16.  How are the MAP tests testing different standards for all states? Are the standards-based tests really based on each state’s individual standards?

o   Yes.  To illustrate, all reading tests developed by NWEA come from a very large pool of items that are all calibrated to a single scale.  The item pool for each state test is selected from this large pool of items and items are chosen to align with each state’s particular content standards.  The reason that tests can be compared across states is that all tests are children of the same parent. 

17.  What kind of data is available among high schools?

o   Although we have more data for grades 3-8, we do have many districts testing in all grades from Kindergarten through grade 12. There are specific challenges in working with high school data which we will communicate to finalists as they develop their Scope of Work.

18.  Can NWEA link to NCES restricted data as well as NCES public data?

o   We do not have access to the restricted data.

19.  What is your definition of an unfunded faculty researcher?

o   One of our goals in establishing the Data Award program was to make the resources of the Kingsbury Center available to researchers who do not have grant funding.  We like grants (who doesn’t), but we believe that education research today is heavily driven by the agenda of external funders, in particular foundations and the federal government.  The fact that funding is available from these groups is a good thing, but we also believe there are researchers who do excellent work who haven’t been funded by grants.  We want to make sure the resources of the Center are available to researchers who may not have grant funding.  So what we mean by that is a faculty researcher who has an unfunded project – a research project that does not have grant funding or designated departmental funds supporting it.  A faculty researcher who has funding for other projects, unrelated to the proposal, can still be funded.